-- Logs begin at Wed 2022-09-21 04:48:51 UTC, end at Wed 2022-09-21 06:06:18 UTC. -- Sep 21 05:12:53.039745 dv-cinder-849143-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 21 05:12:53.047018 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 21 05:12:53.049660 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 21 05:12:53 2022] *** Sep 21 05:12:53.049660 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 21 05:12:53.049660 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: nodename: dv-cinder-849143-5-iscsi Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: machine: x86_64 Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: clock source: unix Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: pcre jit disabled Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: detected number of CPU cores: 4 Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: current working directory: / Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: detected binary path: /usr/bin/uwsgi-core Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: your processes number limit is 47680 Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: your memory page size is 4096 bytes Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: detected max file descriptor number: 2048 Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: lock engine: pthread robust mutexes Sep 21 05:12:53.049881 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: thunder lock: enabled Sep 21 05:12:53.051169 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 21 05:12:53.051264 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 21 05:12:53.092270 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: Python main interpreter initialized at 0x55a260cff9c0 Sep 21 05:12:53.092270 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: python threads support enabled Sep 21 05:12:53.092270 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: your server socket listen backlog is limited to 100 connections Sep 21 05:12:53.092270 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: your mercy for graceful operations on workers is 90 seconds Sep 21 05:12:53.092662 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: mapped 403077 bytes (393 KB) for 2 cores Sep 21 05:12:53.092744 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: *** Operational MODE: preforking *** Sep 21 05:12:53.092973 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: *** uWSGI is running in multiple interpreter mode *** Sep 21 05:12:53.092973 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: spawned uWSGI master process (pid: 81395) Sep 21 05:12:53.093181 dv-cinder-849143-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 21 05:12:53.095486 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: spawned uWSGI worker 1 (pid: 81410, cores: 1) Sep 21 05:12:53.095486 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: spawned uWSGI worker 2 (pid: 81411, cores: 1) Sep 21 05:12:53.095486 dv-cinder-849143-5-iscsi devstack@keystone.service[81395]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 21 05:12:57.377960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 05:12:57.377960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: from cryptography.utils import int_from_bytes Sep 21 05:12:57.378767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 05:12:57.378767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: from cryptography.utils import int_from_bytes Sep 21 05:12:57.379205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 05:12:57.379205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: from cryptography.utils import int_from_bytes Sep 21 05:12:57.379341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 05:12:57.379341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: from cryptography.utils import int_from_bytes Sep 21 05:12:57.427925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 21 05:12:57.428067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 21 05:12:57.428163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 21 05:12:57.428254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 21 05:12:57.428334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] command line args: [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 21 05:12:57.428406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] command line args: [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 21 05:12:57.428541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 21 05:12:57.428671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 21 05:12:57.428758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ================================================================================ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 21 05:12:57.428834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ================================================================================ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 21 05:12:57.428954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] admin_token = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] admin_token = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] config_dir = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] config_dir = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] config_source = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] config_source = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] debug = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] debug = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.429962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.430192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.430294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.430362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.430499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.430576 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.430725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.430811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.430974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_config_append = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_config_append = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_dir = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_dir = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_file = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.431958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_file = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.432114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.432194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.432360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.432446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.432606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.432712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.432850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.432958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.433021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.433085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.433248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.433369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.433451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.433520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.433664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.433768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.433892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.434059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.434137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.434202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.434362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.434470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.434594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.434700 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.434839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.434944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.435084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.435172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.435353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.435447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.435832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.435919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] publish_errors = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] publish_errors = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.436960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.437092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.437171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.437326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] standard_threads = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.437453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] standard_threads = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438432 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_journal = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438828 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_journal = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_json = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.438983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_json = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.439054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_stderr = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.439173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_stderr = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.439257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_syslog = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.439338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_syslog = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.439412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.439488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 05:12:57.439641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.439721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.439878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.439965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.440133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.440213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.440369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.440459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.440615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.440706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.440852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.440931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.441101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.441179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.441335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.441421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.441580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.441659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.441826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.441903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.442095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.442191 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.442328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.442413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.442570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.442656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.442805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.442892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.443049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.443128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.443292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.443371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.443525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.443613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.443776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.443854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.444027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.444115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.444265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.444347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.444492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.444577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.444742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.444827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.445004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.445089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.445252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.445331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.445499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.445583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.445730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.445809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.445983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.446078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.446207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.446286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.446442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.446522 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.446687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.446766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.446929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.447008 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.447158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.447239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.447386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.447465 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.447629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.447723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.447868 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.447956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.448101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.448181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.448349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.448432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.448575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.448654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.448806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.448884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.449036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.449114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.449265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.449345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.449493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.449575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.449719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.449797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.449958 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.450034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.450189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.450270 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.450432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.450511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.450658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.450738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.450891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.450974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.451122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.451205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.451355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.451435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.451588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.451666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.451956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452088 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.452955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.453109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.453239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.453327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.453445 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.453520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.453636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.453711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.453851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.453973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.454108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.454214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.454339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.454436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.454593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.454682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.454827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.454924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.455057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.455182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.455318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.455408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.455519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.455608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.455781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.455862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.456017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.456099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.456264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.456362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.456505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.456608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.456742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.456836 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.456977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.457073 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.457211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.457289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.457452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.457537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.457687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.457781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.457919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.458049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.458170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.458246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.458398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.458490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.458628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.458705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.458862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.458945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.459095 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.459183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.459326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.459404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.459556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.459642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.459789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.459875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.460043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.460166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.460281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.460400 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.460516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.460600 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.460749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.460830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.460984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.461068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.461219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.461374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.461450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.461521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.461672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.461750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.461903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.462023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.462153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.462235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.462381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.462461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.462608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.462687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.462840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.462920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.463066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.463145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.463303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.463383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.463534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.463615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.463778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.463864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.464009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.464087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.464260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.464350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.464515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.464596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.464754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.464859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.464980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.external = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.465061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.external = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.465209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.465299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.465500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.465580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.465727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.465806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.465992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.password = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.466079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.password = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.466203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.token = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.466284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.token = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.466436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.466516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.466668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.466751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.466910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.466989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.467138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.467217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.467402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.467490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.467621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.467702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.467848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.467928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.468079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.468161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.468321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.468401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.468565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.468645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.469334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.469427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.469581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.469662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.469833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.469913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.470097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.470183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.470349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.470430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.470606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.470686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.470847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.470928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.471097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.471178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.471378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.471467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.471620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.471701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.471880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.471962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.472118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.472198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.472371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.472453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.472603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.472683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.472836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.472915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.473077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.473159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.473319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.473399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.473548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.473635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.473789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.473869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.474045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.474138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.474298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.474388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.474532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.474612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.474784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.474863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.475015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.475103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.475259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.475339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.475492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.475571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.475736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.475893 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.475970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.476043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.476201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.476282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.476434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.476522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.476676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.476755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.476903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.476982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.477136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.477224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.477382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.477461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.477628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.477707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.477858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.477966 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.478109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.478190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.478340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.478426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.478583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.478671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.478827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.478916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.479071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.479152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.479324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.479413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.479569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.479649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.479833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.479917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.480067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.480157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.480299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.480380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.480527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.480608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.480759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.480848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.480986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.481067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.481218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.481298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.481457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.481545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.481692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.481771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.481980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.482065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.482188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.482270 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.482433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.482513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.482675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.482754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.482907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.482989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.483150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.483236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.483391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.483471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.483619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.483701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.483851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.483931 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.484079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.484160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.484314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.485023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.485162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.485245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.485409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.485497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.485647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.485730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.485883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.485981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.486158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.486238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.486508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.486591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.486742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.486822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.486974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.487053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.487223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.487306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.487470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.487550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.487707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.487777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.487939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.488278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.488427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.488507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.488659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.488737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.488896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.488977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.489152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.489233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.489383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.489464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.489631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.489711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.489873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.490023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.490142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.490223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.490376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.490459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.490605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.490685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.490839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.490919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.491069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.491151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.491302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.491381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.491533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.491612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.491779 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.491859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.492018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.492097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.492261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.492342 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.492503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.492585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.492745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.492824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.492972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.493051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.493214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.493296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.493447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.493528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.493680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.493760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.493913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.494013 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.494165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.494246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.494408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.494488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.494651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.494733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.494885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.494965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.495133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.495213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.495363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.495448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.495601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.495682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.495833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.495914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.496067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.496148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.496304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.496415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.496544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.496627 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.496783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.496865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.497011 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.497089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.497245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.497323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.497481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.497561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.497727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.497807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.497993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.498075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.498230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.498310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.498476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.498554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.498721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.498808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.498953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.499033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.499196 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.499275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.499431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.499585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.499662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.499733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.499883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] role.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.499971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] role.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.500161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] role.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.500251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] role.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.500396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.500478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.500639 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.500717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.500883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.500973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.501126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.501207 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.501369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.501448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.501614 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.501707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.501855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.501965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.502117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.502197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.502360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.502452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.502587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.502667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.502830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.502909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.503073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.503164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.503315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.503395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.503557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.503636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.503938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.504026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.504171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.504256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.504416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.504544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.504661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.504754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.504905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.504985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.505148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.505228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.505380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.505461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.505627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.505706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.505873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.505971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.506133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.506216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.506361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.506440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.506591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.506669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.506818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.506898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.507063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.507142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.507312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.507391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.507557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.507637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.507787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.507866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.508020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.508099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.508249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.508330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.508480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.508559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.508714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.508792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.508956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.509035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.509190 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.509269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.509444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.509522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.509688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.509767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.509922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.510020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.510182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.510268 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.510422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.510504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.510666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.510745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.510900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.510978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.511132 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.511212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.511366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.511444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.511609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.511686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.511876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.511961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.512104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.512192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.512343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.512436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.512601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.512682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.512854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.512942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.513094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.513173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.513334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.513412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.513568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.513656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.513805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.513885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.514057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.514139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.514290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.514383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.514523 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.514666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.514744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.514816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.514979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.515069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.515211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.515296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.515447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.515529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.515700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.515788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.515945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.516026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.516178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.516257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.516413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.516501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.516642 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.516723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.516879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.516958 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.517108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.517195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 05:12:57.517270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 21 05:12:57.517343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 21 05:12:57.546609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.546747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.546838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.546919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.547611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.547708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.555921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.556094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.583743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.583911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.584018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.584120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.638069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.638374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.686757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.686990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.687282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.687553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.842346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.842531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.842604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.842681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.842780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.842860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.842999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.843084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.843217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.843311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.843433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.843517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.866943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.867131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.867253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.867380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.867508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.867590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.867713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.867798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.867882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.868005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.868089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.868218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.868299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.868431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.879416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.879553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.892552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.892734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.892823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.892960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.893045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.893180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.893263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.893396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.893486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.893610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.893692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.893820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.893901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.894066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 05:12:57.910943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.911156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 05:12:57.911491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.911577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.911693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 05:12:57.911793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 05:12:57.911972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.912055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.912211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 05:12:57.912292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.912481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.912564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.912735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.912814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.912991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.913247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.922304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.922585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.922783 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.923061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.923164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.923334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.923618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.930101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.930570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.933000 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.933291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.933449 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.933582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.933733 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.933867 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.934047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.934180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.934306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.934446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.934593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.934717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.934855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.934990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.935121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.935261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.935397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.935535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.935666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.935807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.935936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.936203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.958988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.959314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.960448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.960737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.960818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.961038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.965416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.965705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.965786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.966012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.966107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.966294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.972885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.973176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.973264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.973474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.978753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.979045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:57.979129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.979330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.979411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.979615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.979697 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.979894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.979974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.980170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.980260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.980442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.980588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.980712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.980792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.980984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.996485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.996768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.996849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:57.997058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.000665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.000960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.003939 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.004227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.004311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.004522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.008459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.008742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.011611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.011932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.014860 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.015147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.015232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.015436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.015519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.016235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.016324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.016516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.016587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.016831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.016910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.017121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.033215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.033510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.033601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.033784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.033876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.034076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.034167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.034344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.040929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.041162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.042484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.042722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.044025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.044226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.044317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.044515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.047063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.047260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.047352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.047545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.047637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.047827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.047919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.048118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.048285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.048517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.059491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.059645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.059783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.059930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.060066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.060210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.060351 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.060491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.060734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.060865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.061027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.061161 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.061318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.061437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.076916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.077024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.080383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.080477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.083301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.083386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.084810 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.084893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.086299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.086383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.086583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.086665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.086981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.087063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.093314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.093402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.095370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.095479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.098261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.098351 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.098657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.098740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.098945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.099035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.099222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.099307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.107016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.107102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.107308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.107399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.107589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.107670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.116908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.117403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.117755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.118063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.118343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.118626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.118901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.118993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 05:12:58.119184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.119306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.119454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.119535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.119740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.119830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.120020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.120102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.120279 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.120360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.120543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.120623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.120880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.121021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.121219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.121514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.121800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.122093 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.122364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.122688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 05:12:58.155434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a260cff9c0 pid: 81411 (default app) Sep 21 05:12:58.157742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a260cff9c0 pid: 81410 (default app) Sep 21 05:12:58.159264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66fc12-1ef7-4381-8a35-d857157ad872 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:12:58.159556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66fc12-1ef7-4381-8a35-d857157ad872 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:12:58.159825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66fc12-1ef7-4381-8a35-d857157ad872 None None] PATH_INFO: `/v3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:12:58.163928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Wed Sep 21 05:12:58 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:04.905357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca40606-a8f1-49a1-922a-3b0f83994ef9 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:04.905844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca40606-a8f1-49a1-922a-3b0f83994ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:04.905844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca40606-a8f1-49a1-922a-3b0f83994ef9 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:04.911646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:04.916900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:04.917398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:04.917492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:04.988412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: INFO dbcounter [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] Registered counter for database keystone Sep 21 05:13:05.006909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG oslo_db.sqlalchemy.engines [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81411) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 21 05:13:05.012954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writer thread running {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 21 05:13:05.199303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG passlib.handlers.bcrypt [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81411) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 21 05:13:05.199703 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG passlib.handlers.bcrypt [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81411) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 21 05:13:05.261016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:05.263127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:05.401462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 21 05:13:05.888043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca40606-a8f1-49a1-922a-3b0f83994ef9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:05.961296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: INFO dbcounter [None req-7ca40606-a8f1-49a1-922a-3b0f83994ef9 None None] Registered counter for database keystone Sep 21 05:13:05.984036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG oslo_db.sqlalchemy.engines [None req-7ca40606-a8f1-49a1-922a-3b0f83994ef9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81410) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 21 05:13:05.991263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writer thread running {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 21 05:13:06.115425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca40606-a8f1-49a1-922a-3b0f83994ef9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:06.119640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667d675-08d3-4c03-b94d-34c847a05de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:06.120277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667d675-08d3-4c03-b94d-34c847a05de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:06.120561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667d675-08d3-4c03-b94d-34c847a05de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:06.120834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667d675-08d3-4c03-b94d-34c847a05de1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:06.129516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667d675-08d3-4c03-b94d-34c847a05de1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:06.185070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0667d675-08d3-4c03-b94d-34c847a05de1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 21 05:13:06.185365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667d675-08d3-4c03-b94d-34c847a05de1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:06.195513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Sep 21 05:13:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 05:13:06.201439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:06.217894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f77384-e5b5-4eea-a214-3127b7e1fa66 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:06.222310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f38d1-24f4-4a70-a6f2-a21f3561848a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:06.223426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f38d1-24f4-4a70-a6f2-a21f3561848a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:06.223714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f38d1-24f4-4a70-a6f2-a21f3561848a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:06.223996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f38d1-24f4-4a70-a6f2-a21f3561848a admin admin] PATH_INFO: `/v3/projects/01b2fa2971c94d418f0122cbda3c72f9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:06.225605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f38d1-24f4-4a70-a6f2-a21f3561848a admin admin] RBAC: Authorizing `identity:get_project(project_id=01b2fa2971c94d418f0122cbda3c72f9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:06.279523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING keystone.common.rbac_enforcer.enforcer [None req-046f38d1-24f4-4a70-a6f2-a21f3561848a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 21 05:13:06.279876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f38d1-24f4-4a70-a6f2-a21f3561848a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:06.284365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Sep 21 05:13:06 2022] GET /identity/v3/projects/01b2fa2971c94d418f0122cbda3c72f9 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:08.370990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1efb3-d5e2-4b58-baf9-f7b98e5b9093 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:08.371535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1efb3-d5e2-4b58-baf9-f7b98e5b9093 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:08.371535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1efb3-d5e2-4b58-baf9-f7b98e5b9093 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:08.376223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:08.381564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67197da2-2c04-4f6d-869d-f6061c47794a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:08.381980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67197da2-2c04-4f6d-869d-f6061c47794a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:08.382158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67197da2-2c04-4f6d-869d-f6061c47794a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:08.401893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-67197da2-2c04-4f6d-869d-f6061c47794a None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:08.403437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-67197da2-2c04-4f6d-869d-f6061c47794a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:08.411189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 21 05:13:08.681173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1efb3-d5e2-4b58-baf9-f7b98e5b9093 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:08.693666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1efb3-d5e2-4b58-baf9-f7b98e5b9093 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:08.698093 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb78ef-a4da-4ec1-82c6-e3240b8341ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:08.698800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb78ef-a4da-4ec1-82c6-e3240b8341ef admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:08.699088 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb78ef-a4da-4ec1-82c6-e3240b8341ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:08.699363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb78ef-a4da-4ec1-82c6-e3240b8341ef admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:08.707019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb78ef-a4da-4ec1-82c6-e3240b8341ef admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:08.708776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb78ef-a4da-4ec1-82c6-e3240b8341ef admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:08.718050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Sep 21 05:13:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:13:08.723920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67197da2-2c04-4f6d-869d-f6061c47794a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:08.732704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67197da2-2c04-4f6d-869d-f6061c47794a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:08.737593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a8685-8442-4a41-9567-f2e421af4a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:08.738496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a8685-8442-4a41-9567-f2e421af4a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:08.738773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a8685-8442-4a41-9567-f2e421af4a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:08.739044 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a8685-8442-4a41-9567-f2e421af4a5c admin admin] PATH_INFO: `/v3/users/493af8396fb3419ea552b2a7642f7a68` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:08.740332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a8685-8442-4a41-9567-f2e421af4a5c admin admin] RBAC: Authorizing `identity:get_user(user_id=493af8396fb3419ea552b2a7642f7a68)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:08.742169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a8685-8442-4a41-9567-f2e421af4a5c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:08.743655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Wed Sep 21 05:13:08 2022] GET /identity/v3/users/493af8396fb3419ea552b2a7642f7a68 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:10.946381 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b03a0-c7ba-41b3-99a9-153f74e3ac78 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:10.946894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b03a0-c7ba-41b3-99a9-153f74e3ac78 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:10.946894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b03a0-c7ba-41b3-99a9-153f74e3ac78 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:10.948155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:10.953372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3232a-e706-4af8-a163-29d6f996124f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:10.953782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3232a-e706-4af8-a163-29d6f996124f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:10.953949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3232a-e706-4af8-a163-29d6f996124f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:10.972041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6af3232a-e706-4af8-a163-29d6f996124f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:10.973210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6af3232a-e706-4af8-a163-29d6f996124f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:10.978048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb6d9e-9e4f-4dae-954a-1a7a70d2b2ef None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:10.978333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb6d9e-9e4f-4dae-954a-1a7a70d2b2ef None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:10.978605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb6d9e-9e4f-4dae-954a-1a7a70d2b2ef None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:10.979903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:10.985700 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55f213-3983-4c6a-9470-15a6e9f1c480 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:10.986083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55f213-3983-4c6a-9470-15a6e9f1c480 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:10.986365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55f213-3983-4c6a-9470-15a6e9f1c480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:11.027180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG passlib.handlers.bcrypt [None req-9b55f213-3983-4c6a-9470-15a6e9f1c480 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81410) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 21 05:13:11.027558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG passlib.handlers.bcrypt [None req-9b55f213-3983-4c6a-9470-15a6e9f1c480 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81410) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 21 05:13:11.054524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:11.076767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9b55f213-3983-4c6a-9470-15a6e9f1c480 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:11.078401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9b55f213-3983-4c6a-9470-15a6e9f1c480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:11.177763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:13:11.319055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3232a-e706-4af8-a163-29d6f996124f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:11.342933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:11.423333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3232a-e706-4af8-a163-29d6f996124f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:11.443475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55f213-3983-4c6a-9470-15a6e9f1c480 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:11.456396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978846-6ecf-47f3-81f1-9fa9d052de0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:11.457031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978846-6ecf-47f3-81f1-9fa9d052de0c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:11.457311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978846-6ecf-47f3-81f1-9fa9d052de0c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:11.457590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978846-6ecf-47f3-81f1-9fa9d052de0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:11.469215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e978846-6ecf-47f3-81f1-9fa9d052de0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-2e978846-6ecf-47f3-81f1-9fa9d052de0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:11.478025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:11.479126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:11.480007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:13:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:11.484829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978846-6ecf-47f3-81f1-9fa9d052de0c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:11.546102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55f213-3983-4c6a-9470-15a6e9f1c480 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:11.548388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:11.568339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:11.586246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880addf8-0676-47aa-b8b6-301c853f4c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:11.586246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880addf8-0676-47aa-b8b6-301c853f4c5b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:11.586246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880addf8-0676-47aa-b8b6-301c853f4c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:11.586246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880addf8-0676-47aa-b8b6-301c853f4c5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:11.592577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978846-6ecf-47f3-81f1-9fa9d052de0c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:11.627457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10355-3188-4154-b26b-a79524f142d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:11.628066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10355-3188-4154-b26b-a79524f142d0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:11.628342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10355-3188-4154-b26b-a79524f142d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:11.628614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10355-3188-4154-b26b-a79524f142d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:11.629288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10355-3188-4154-b26b-a79524f142d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:11.629595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10355-3188-4154-b26b-a79524f142d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:11.631121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10355-3188-4154-b26b-a79524f142d0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:11.643137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:11.648939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10355-3188-4154-b26b-a79524f142d0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:11.654190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880addf8-0676-47aa-b8b6-301c853f4c5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:11.655719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880addf8-0676-47aa-b8b6-301c853f4c5b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:11.745724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10355-3188-4154-b26b-a79524f142d0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:11.751750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 05:13:11.757203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880addf8-0676-47aa-b8b6-301c853f4c5b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:11.759836 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:11.779843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ea9d7-8d41-430c-8126-f3d260281568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:11.780477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ea9d7-8d41-430c-8126-f3d260281568 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:11.780782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ea9d7-8d41-430c-8126-f3d260281568 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:11.781087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ea9d7-8d41-430c-8126-f3d260281568 None admin] PATH_INFO: `/v3/users/493af8396fb3419ea552b2a7642f7a68` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:11.782294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ea9d7-8d41-430c-8126-f3d260281568 None admin] RBAC: Authorizing `identity:get_user(user_id=493af8396fb3419ea552b2a7642f7a68)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:11.783747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ea9d7-8d41-430c-8126-f3d260281568 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:11.785305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:11 2022] GET /identity/v3/users/493af8396fb3419ea552b2a7642f7a68 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:11.790570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ea9d7-8d41-430c-8126-f3d260281568 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:11.804121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:11.856560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880addf8-0676-47aa-b8b6-301c853f4c5b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:11.889702 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ea9d7-8d41-430c-8126-f3d260281568 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:11.890099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ab306-b259-4ae7-9866-50133b4397aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:11.891476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ab306-b259-4ae7-9866-50133b4397aa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:11.891787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ab306-b259-4ae7-9866-50133b4397aa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:11.892059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ab306-b259-4ae7-9866-50133b4397aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:11.910020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ab306-b259-4ae7-9866-50133b4397aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:11.911480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ab306-b259-4ae7-9866-50133b4397aa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:11.923810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a1037-f647-4087-ac8b-1a66f9e0aed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:11.924431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a1037-f647-4087-ac8b-1a66f9e0aed8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:11.924713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a1037-f647-4087-ac8b-1a66f9e0aed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:11.924984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a1037-f647-4087-ac8b-1a66f9e0aed8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:11.926104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a1037-f647-4087-ac8b-1a66f9e0aed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:11.927532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a1037-f647-4087-ac8b-1a66f9e0aed8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:11.929048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ef7ab306-b259-4ae7-9866-50133b4397aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:13:11.929451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:11.930346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:13:11.931186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:13:11.931186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:13:11.931186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:11.931186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:13:11.935029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a1037-f647-4087-ac8b-1a66f9e0aed8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:11.936080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ab306-b259-4ae7-9866-50133b4397aa None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:11.990455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:12.014185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:12.033908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a1037-f647-4087-ac8b-1a66f9e0aed8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:12.036409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ab306-b259-4ae7-9866-50133b4397aa None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:12.067051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c036bd1-8e16-44f5-901f-80500dd1f651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:12.067653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036bd1-8e16-44f5-901f-80500dd1f651 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:12.067929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036bd1-8e16-44f5-901f-80500dd1f651 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:12.068199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036bd1-8e16-44f5-901f-80500dd1f651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:12.068730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5c6ca-8cd9-4499-b13d-87f8452c8307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:12.068941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c036bd1-8e16-44f5-901f-80500dd1f651 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.domain.id=default, user.id=493af8396fb3419ea552b2a7642f7a68) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:12.069241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c036bd1-8e16-44f5-901f-80500dd1f651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:12.069344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5c6ca-8cd9-4499-b13d-87f8452c8307 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:12.069599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5c6ca-8cd9-4499-b13d-87f8452c8307 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:12.069869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5c6ca-8cd9-4499-b13d-87f8452c8307 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:12.070528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5c6ca-8cd9-4499-b13d-87f8452c8307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:12.070684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c036bd1-8e16-44f5-901f-80500dd1f651 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:12.070831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5c6ca-8cd9-4499-b13d-87f8452c8307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:12.072189 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5c6ca-8cd9-4499-b13d-87f8452c8307 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:12.077173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Sep 21 05:13:11 2022] GET /identity/v3/role_assignments?user.id=493af8396fb3419ea552b2a7642f7a68&scope.domain.id=default&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:12.101972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:14.414428 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc5509-ef2c-461c-9d44-d0c8733493bb None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:14.414933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc5509-ef2c-461c-9d44-d0c8733493bb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:14.415005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc5509-ef2c-461c-9d44-d0c8733493bb None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:14.416585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:14.422139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fd71-e749-4ce1-add2-6851cc84903c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:14.422547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fd71-e749-4ce1-add2-6851cc84903c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:14.422736 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fd71-e749-4ce1-add2-6851cc84903c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:14.451368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6499fd71-e749-4ce1-add2-6851cc84903c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:14.452625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6499fd71-e749-4ce1-add2-6851cc84903c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:14.539255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 11/23] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:14.798990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e6518-bf00-43da-9896-76af2bb9fa23 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:14.799187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e6518-bf00-43da-9896-76af2bb9fa23 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:14.799612 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e6518-bf00-43da-9896-76af2bb9fa23 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:14.801342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 13/24] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:14.806398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4dd74-53a0-4eea-8c2b-c291e48dd86c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:14.806692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4dd74-53a0-4eea-8c2b-c291e48dd86c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:14.806977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4dd74-53a0-4eea-8c2b-c291e48dd86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:14.835005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5ff4dd74-53a0-4eea-8c2b-c291e48dd86c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:14.836223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5ff4dd74-53a0-4eea-8c2b-c291e48dd86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:14.893457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:14.932240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 12/25] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:15.045549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e6518-bf00-43da-9896-76af2bb9fa23 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:15.134943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118eac2-1f6e-432a-afe3-d0d2fe9dd0c3 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:15.135598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118eac2-1f6e-432a-afe3-d0d2fe9dd0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:15.136074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118eac2-1f6e-432a-afe3-d0d2fe9dd0c3 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:15.137588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 13/26] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:15.143173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e459c-4416-4744-9184-7db99bfe6276 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:15.143676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e459c-4416-4744-9184-7db99bfe6276 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:15.144167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e459c-4416-4744-9184-7db99bfe6276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:15.175934 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-441e459c-4416-4744-9184-7db99bfe6276 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:15.177369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-441e459c-4416-4744-9184-7db99bfe6276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:15.192192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e6518-bf00-43da-9896-76af2bb9fa23 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:15.257168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad2f0e-5a3d-45c2-9d57-8259a1e7f3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:15.258055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad2f0e-5a3d-45c2-9d57-8259a1e7f3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:15.258471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad2f0e-5a3d-45c2-9d57-8259a1e7f3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:15.258949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad2f0e-5a3d-45c2-9d57-8259a1e7f3f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:15.280426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad2f0e-5a3d-45c2-9d57-8259a1e7f3f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-e0ad2f0e-5a3d-45c2-9d57-8259a1e7f3f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:15.291727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:15.292759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:15.293660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:15.297281 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 14/27] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:13:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:15.299126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4545b-9ab9-47ed-8977-41baf3088ae3 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:15.300455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 14/28] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 21 05:13:15.300903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4545b-9ab9-47ed-8977-41baf3088ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:15.301315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4545b-9ab9-47ed-8977-41baf3088ae3 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:15.302819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 15/29] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:15.310369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ae4df-b134-44d5-b896-305158b1e370 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:15.310814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ae4df-b134-44d5-b896-305158b1e370 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:15.311246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ae4df-b134-44d5-b896-305158b1e370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:15.314869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4545b-9ab9-47ed-8977-41baf3088ae3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:15.322945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:15.350943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-916ae4df-b134-44d5-b896-305158b1e370 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:15.352391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-916ae4df-b134-44d5-b896-305158b1e370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:15.434568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 15/30] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:15.435305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ae4df-b134-44d5-b896-305158b1e370 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:15.496859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:15.558631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4545b-9ab9-47ed-8977-41baf3088ae3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:15.613826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ae4df-b134-44d5-b896-305158b1e370 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:15.639618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff9f71-f4b5-4ca4-9634-7d158b640c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:15.640826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff9f71-f4b5-4ca4-9634-7d158b640c11 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:15.641173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff9f71-f4b5-4ca4-9634-7d158b640c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:15.641447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff9f71-f4b5-4ca4-9634-7d158b640c11 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:15.642254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff9f71-f4b5-4ca4-9634-7d158b640c11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:15.642558 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff9f71-f4b5-4ca4-9634-7d158b640c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:15.644101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff9f71-f4b5-4ca4-9634-7d158b640c11 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:15.660392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 16/31] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:15.666143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff9f71-f4b5-4ca4-9634-7d158b640c11 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:15.680424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de5de5-2559-4c23-9308-045225040c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:15.681046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de5de5-2559-4c23-9308-045225040c13 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:15.681324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de5de5-2559-4c23-9308-045225040c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:15.681593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de5de5-2559-4c23-9308-045225040c13 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:15.682284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de5de5-2559-4c23-9308-045225040c13 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:15.683665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de5de5-2559-4c23-9308-045225040c13 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:15.779108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:15.833060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff9f71-f4b5-4ca4-9634-7d158b640c11 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:15.862133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 16/32] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Sep 21 05:13:15 2022] POST /identity/v3/roles => generated 219 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:15.863691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de5de5-2559-4c23-9308-045225040c13 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:15.877969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10852637-6b26-42a6-93ac-53a7c7f85df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:15.877969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10852637-6b26-42a6-93ac-53a7c7f85df2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:15.877969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10852637-6b26-42a6-93ac-53a7c7f85df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:15.877969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10852637-6b26-42a6-93ac-53a7c7f85df2 None admin] PATH_INFO: `/v3/users/493af8396fb3419ea552b2a7642f7a68` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:15.886942 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10852637-6b26-42a6-93ac-53a7c7f85df2 None admin] RBAC: Authorizing `identity:get_user(user_id=493af8396fb3419ea552b2a7642f7a68)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:15.888479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10852637-6b26-42a6-93ac-53a7c7f85df2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:15.895815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 17/33] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:15 2022] GET /identity/v3/users/493af8396fb3419ea552b2a7642f7a68 => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:15.896766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10852637-6b26-42a6-93ac-53a7c7f85df2 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:16.029448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:16.037971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de5de5-2559-4c23-9308-045225040c13 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:16.090516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd224a-aab5-4124-93bd-01518eaeddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:16.091322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd224a-aab5-4124-93bd-01518eaeddfe None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.091970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd224a-aab5-4124-93bd-01518eaeddfe None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.092259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd224a-aab5-4124-93bd-01518eaeddfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.094658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10852637-6b26-42a6-93ac-53a7c7f85df2 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:16.106924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd224a-aab5-4124-93bd-01518eaeddfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:16.109888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd224a-aab5-4124-93bd-01518eaeddfe None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-0edd224a-aab5-4124-93bd-01518eaeddfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:13:16.132059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:16.132947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:13:16.133714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:13:16.133714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:13:16.133714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:16.133877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 17/34] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:16.134890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd224a-aab5-4124-93bd-01518eaeddfe None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:16.147850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8083089-22ea-4c19-925b-5d1b3ae05954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:16.148514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8083089-22ea-4c19-925b-5d1b3ae05954 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.148797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8083089-22ea-4c19-925b-5d1b3ae05954 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.149065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8083089-22ea-4c19-925b-5d1b3ae05954 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.149740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8083089-22ea-4c19-925b-5d1b3ae05954 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:16.151222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8083089-22ea-4c19-925b-5d1b3ae05954 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:16.158734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:16.259792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd224a-aab5-4124-93bd-01518eaeddfe None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:16.304066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 18/35] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Sep 21 05:13:15 2022] POST /identity/v3/roles => generated 225 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:16.305044 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8083089-22ea-4c19-925b-5d1b3ae05954 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:16.306414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5554c-09f1-4697-a6a1-898505c982e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:16.307300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5554c-09f1-4697-a6a1-898505c982e4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.307795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5554c-09f1-4697-a6a1-898505c982e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.308634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5554c-09f1-4697-a6a1-898505c982e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.311682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5554c-09f1-4697-a6a1-898505c982e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:16.313632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5554c-09f1-4697-a6a1-898505c982e4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:16.315687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 18/36] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:16.321918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5554c-09f1-4697-a6a1-898505c982e4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:16.342995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:16.434250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8083089-22ea-4c19-925b-5d1b3ae05954 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:16.452167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5554c-09f1-4697-a6a1-898505c982e4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:16.473976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb9480-b03c-4ed8-8076-3cf6413dfea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:16.474606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9480-b03c-4ed8-8076-3cf6413dfea4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.474887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9480-b03c-4ed8-8076-3cf6413dfea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.475159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9480-b03c-4ed8-8076-3cf6413dfea4 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.475813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb9480-b03c-4ed8-8076-3cf6413dfea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:16.476116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb9480-b03c-4ed8-8076-3cf6413dfea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:16.477665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb9480-b03c-4ed8-8076-3cf6413dfea4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:16.494087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9819b6-a605-4b8c-9370-386cab2a016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:16.494773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9819b6-a605-4b8c-9370-386cab2a016d None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.495061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9819b6-a605-4b8c-9370-386cab2a016d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.495490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9819b6-a605-4b8c-9370-386cab2a016d None admin] PATH_INFO: `/v3/domains/default/users/493af8396fb3419ea552b2a7642f7a68/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.497824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 19/37] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:16.497991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9819b6-a605-4b8c-9370-386cab2a016d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=493af8396fb3419ea552b2a7642f7a68, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:16.499558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9819b6-a605-4b8c-9370-386cab2a016d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:16.505960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632b9d5-debc-4b3f-ad7a-cccde17b0a9c None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.505960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632b9d5-debc-4b3f-ad7a-cccde17b0a9c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.505960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632b9d5-debc-4b3f-ad7a-cccde17b0a9c None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.505960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 20/38] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:16.507392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26d3c2-6629-496e-a1f9-b9673c2adc32 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.507700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26d3c2-6629-496e-a1f9-b9673c2adc32 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.507992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26d3c2-6629-496e-a1f9-b9673c2adc32 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.509522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 21/39] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:16.510649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26d3c2-6629-496e-a1f9-b9673c2adc32 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:16.525074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 19/40] 10.222.0.18 () {58 vars in 1361 bytes} [Wed Sep 21 05:13:16 2022] PUT /identity/v3/domains/default/users/493af8396fb3419ea552b2a7642f7a68/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:13:16.527096 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19440b2b-c2f4-480b-9231-a49bb647f61b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.527616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19440b2b-c2f4-480b-9231-a49bb647f61b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.528067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19440b2b-c2f4-480b-9231-a49bb647f61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.551395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-19440b2b-c2f4-480b-9231-a49bb647f61b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:16.553000 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-19440b2b-c2f4-480b-9231-a49bb647f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:16.590257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:16.656785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 20/41] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:16.662057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa3d16-2e77-4371-a9af-0094a07bc0ed None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.662057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa3d16-2e77-4371-a9af-0094a07bc0ed None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.662057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa3d16-2e77-4371-a9af-0094a07bc0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.664519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26d3c2-6629-496e-a1f9-b9673c2adc32 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:16.673031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:16.682898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-befa3d16-2e77-4371-a9af-0094a07bc0ed None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:16.685024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-befa3d16-2e77-4371-a9af-0094a07bc0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:16.714843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff044ce-52b6-404e-8425-480cffddd635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:16.715467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff044ce-52b6-404e-8425-480cffddd635 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:16.715746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff044ce-52b6-404e-8425-480cffddd635 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:16.716073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff044ce-52b6-404e-8425-480cffddd635 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:16.716835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff044ce-52b6-404e-8425-480cffddd635 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:16.719146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff044ce-52b6-404e-8425-480cffddd635 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:16.751174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 22/42] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:13:16 2022] POST /identity/v3/projects => generated 301 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:16.766395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 21/43] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:16.944750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff044ce-52b6-404e-8425-480cffddd635 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:17.035102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa3d16-2e77-4371-a9af-0094a07bc0ed None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:17.048018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff044ce-52b6-404e-8425-480cffddd635 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:17.057802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:17.087504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1be589-5d70-45a5-b8ed-850cf1a991c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:17.088142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1be589-5d70-45a5-b8ed-850cf1a991c6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:17.088416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1be589-5d70-45a5-b8ed-850cf1a991c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:17.088685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1be589-5d70-45a5-b8ed-850cf1a991c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:17.089779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1be589-5d70-45a5-b8ed-850cf1a991c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:17.091263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1be589-5d70-45a5-b8ed-850cf1a991c6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:17.093017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 23/44] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:17.098219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1be589-5d70-45a5-b8ed-850cf1a991c6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:17.153664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa3d16-2e77-4371-a9af-0094a07bc0ed None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:17.192684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdcb0e-cbcc-4392-bd22-2a1059dac518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:17.193321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdcb0e-cbcc-4392-bd22-2a1059dac518 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:17.193606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdcb0e-cbcc-4392-bd22-2a1059dac518 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:17.193883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdcb0e-cbcc-4392-bd22-2a1059dac518 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:17.194601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdcb0e-cbcc-4392-bd22-2a1059dac518 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:17.196091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdcb0e-cbcc-4392-bd22-2a1059dac518 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:17.201031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:17.217343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 22/45] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Sep 21 05:13:17 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:13:17.322426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1be589-5d70-45a5-b8ed-850cf1a991c6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:17.359347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c5adf-c548-4389-8ded-95c89eb6cd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:17.359976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c5adf-c548-4389-8ded-95c89eb6cd68 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:17.360259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c5adf-c548-4389-8ded-95c89eb6cd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:17.360531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c5adf-c548-4389-8ded-95c89eb6cd68 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:17.361215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c5adf-c548-4389-8ded-95c89eb6cd68 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:17.362731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c5adf-c548-4389-8ded-95c89eb6cd68 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:17.393048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:13:17 2022] POST /identity/v3/projects => generated 312 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:13:18.796846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d48d-38af-4133-8311-f31e75b274d2 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:18.797377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d48d-38af-4133-8311-f31e75b274d2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:18.797674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d48d-38af-4133-8311-f31e75b274d2 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:18.799025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:18.804152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1ccfb-93d3-479a-b877-95779df5f0d2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:18.804536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1ccfb-93d3-479a-b877-95779df5f0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:18.804834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1ccfb-93d3-479a-b877-95779df5f0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:18.823441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7be1ccfb-93d3-479a-b877-95779df5f0d2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:18.824964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7be1ccfb-93d3-479a-b877-95779df5f0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:18.845109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:18.891798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:19.158272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d48d-38af-4133-8311-f31e75b274d2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:19.262244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d48d-38af-4133-8311-f31e75b274d2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:19.294941 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4ef34-b5db-4abc-b8f2-3338a5bf5cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:19.295577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4ef34-b5db-4abc-b8f2-3338a5bf5cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:19.295863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4ef34-b5db-4abc-b8f2-3338a5bf5cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:19.296703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4ef34-b5db-4abc-b8f2-3338a5bf5cc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:19.308056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4ef34-b5db-4abc-b8f2-3338a5bf5cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-0ea4ef34-b5db-4abc-b8f2-3338a5bf5cc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:19.314209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:19.315111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:19.315834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 24/49] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:13:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:19.321136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1ccfb-93d3-479a-b877-95779df5f0d2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:19.418092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1ccfb-93d3-479a-b877-95779df5f0d2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:19.454232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7b544-fcfe-4bbe-acf0-cc7b10d5ffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:19.454862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b544-fcfe-4bbe-acf0-cc7b10d5ffea None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:19.455145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b544-fcfe-4bbe-acf0-cc7b10d5ffea None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:19.455422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b544-fcfe-4bbe-acf0-cc7b10d5ffea None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:19.456111 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7b544-fcfe-4bbe-acf0-cc7b10d5ffea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:19.456421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7b544-fcfe-4bbe-acf0-cc7b10d5ffea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:19.457859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7b544-fcfe-4bbe-acf0-cc7b10d5ffea None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:19.467284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 26/50] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:19.473545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4ef34-b5db-4abc-b8f2-3338a5bf5cc3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:19.528382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:19.553820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f32e7-7e1d-451b-92cf-d5247e15e197 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:19.554142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f32e7-7e1d-451b-92cf-d5247e15e197 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:19.554422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f32e7-7e1d-451b-92cf-d5247e15e197 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:19.555755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 27/51] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:19.560743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46486c5d-5f38-40c2-a038-d36cc1661673 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:19.561030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46486c5d-5f38-40c2-a038-d36cc1661673 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:19.561316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46486c5d-5f38-40c2-a038-d36cc1661673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:19.574500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4ef34-b5db-4abc-b8f2-3338a5bf5cc3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:19.579305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-46486c5d-5f38-40c2-a038-d36cc1661673 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:19.580497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-46486c5d-5f38-40c2-a038-d36cc1661673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:19.594536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:19.611200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a07e-bd88-404b-8bf9-f925028c43ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:19.611831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a07e-bd88-404b-8bf9-f925028c43ff None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:19.612111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a07e-bd88-404b-8bf9-f925028c43ff None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:19.612386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a07e-bd88-404b-8bf9-f925028c43ff None admin] PATH_INFO: `/v3/users/493af8396fb3419ea552b2a7642f7a68` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:19.613757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434a07e-bd88-404b-8bf9-f925028c43ff None admin] RBAC: Authorizing `identity:get_user(user_id=493af8396fb3419ea552b2a7642f7a68)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:19.615233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434a07e-bd88-404b-8bf9-f925028c43ff None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:19.616782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 25/52] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:19 2022] GET /identity/v3/users/493af8396fb3419ea552b2a7642f7a68 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:19.622130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a07e-bd88-404b-8bf9-f925028c43ff None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:19.650153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 28/53] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:13:19.819839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a07e-bd88-404b-8bf9-f925028c43ff None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:19.842297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:19.853777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af1b4a-73aa-44bc-aa77-27f657d6629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:19.854446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af1b4a-73aa-44bc-aa77-27f657d6629b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:19.854733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af1b4a-73aa-44bc-aa77-27f657d6629b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:19.855015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af1b4a-73aa-44bc-aa77-27f657d6629b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:19.856113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af1b4a-73aa-44bc-aa77-27f657d6629b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:19.857563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af1b4a-73aa-44bc-aa77-27f657d6629b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:19.859160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 26/54] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:19.865231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46486c5d-5f38-40c2-a038-d36cc1661673 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:19.915476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af1b4a-73aa-44bc-aa77-27f657d6629b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:19.962341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46486c5d-5f38-40c2-a038-d36cc1661673 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:19.996168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475afefe-938a-42ec-a665-ee9935cb1639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:19.996772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475afefe-938a-42ec-a665-ee9935cb1639 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:19.997056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475afefe-938a-42ec-a665-ee9935cb1639 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:19.997331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475afefe-938a-42ec-a665-ee9935cb1639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:19.998083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475afefe-938a-42ec-a665-ee9935cb1639 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.domain.id=default, user.id=493af8396fb3419ea552b2a7642f7a68) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:19.998393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475afefe-938a-42ec-a665-ee9935cb1639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:20.000369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475afefe-938a-42ec-a665-ee9935cb1639 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:20.005889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 29/55] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Sep 21 05:13:19 2022] GET /identity/v3/role_assignments?user.id=493af8396fb3419ea552b2a7642f7a68&scope.domain.id=default&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:20.017541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af1b4a-73aa-44bc-aa77-27f657d6629b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:20.053315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30277cc-4cbd-4d05-9261-f9078da52f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:20.053998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30277cc-4cbd-4d05-9261-f9078da52f97 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:20.054290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30277cc-4cbd-4d05-9261-f9078da52f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:20.054572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30277cc-4cbd-4d05-9261-f9078da52f97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:20.055770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30277cc-4cbd-4d05-9261-f9078da52f97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:20.057259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30277cc-4cbd-4d05-9261-f9078da52f97 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:20.058964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 27/56] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:20.065375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475afefe-938a-42ec-a665-ee9935cb1639 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:20.082443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:20.173441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475afefe-938a-42ec-a665-ee9935cb1639 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:20.221170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624e4ad-d4c9-4e84-8886-fa15e5527364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:20.222089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624e4ad-d4c9-4e84-8886-fa15e5527364 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:20.222381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624e4ad-d4c9-4e84-8886-fa15e5527364 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:20.222664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624e4ad-d4c9-4e84-8886-fa15e5527364 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:20.223326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624e4ad-d4c9-4e84-8886-fa15e5527364 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:20.224787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624e4ad-d4c9-4e84-8886-fa15e5527364 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:20.252730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 30/57] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Sep 21 05:13:20 2022] POST /identity/v3/projects => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:22.393914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ee15f-73a9-4660-a59a-bd91b2f7fc58 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:22.394449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ee15f-73a9-4660-a59a-bd91b2f7fc58 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:22.394530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ee15f-73a9-4660-a59a-bd91b2f7fc58 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:22.395818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 28/58] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:22.401039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c1174-1ef0-430a-9fde-f635ea0ca4c4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:22.401457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c1174-1ef0-430a-9fde-f635ea0ca4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:22.401607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c1174-1ef0-430a-9fde-f635ea0ca4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:22.419230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d20c1174-1ef0-430a-9fde-f635ea0ca4c4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:22.420432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d20c1174-1ef0-430a-9fde-f635ea0ca4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:22.477272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:22.486388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 31/59] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:22.755261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ee15f-73a9-4660-a59a-bd91b2f7fc58 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:22.814193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:22.853644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ee15f-73a9-4660-a59a-bd91b2f7fc58 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:22.886536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da226ed-629d-4041-b639-fecbb800f581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:22.887154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da226ed-629d-4041-b639-fecbb800f581 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:22.887439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da226ed-629d-4041-b639-fecbb800f581 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:22.887716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da226ed-629d-4041-b639-fecbb800f581 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:22.888811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da226ed-629d-4041-b639-fecbb800f581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:22.890298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da226ed-629d-4041-b639-fecbb800f581 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:22.891828 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 29/60] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:22.897194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c1174-1ef0-430a-9fde-f635ea0ca4c4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:22.992240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c1174-1ef0-430a-9fde-f635ea0ca4c4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:23.024141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d914632-063b-4e14-a3f2-39d7f8e2bc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:23.024679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d914632-063b-4e14-a3f2-39d7f8e2bc8a None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:23.024962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d914632-063b-4e14-a3f2-39d7f8e2bc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:23.025236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d914632-063b-4e14-a3f2-39d7f8e2bc8a None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:23.025885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d914632-063b-4e14-a3f2-39d7f8e2bc8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:23.027335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d914632-063b-4e14-a3f2-39d7f8e2bc8a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:23.053781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 32/61] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:13:22 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:26.084005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc975aeb-1447-44ff-b48d-0e8f08e6bdb8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:26.085015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc975aeb-1447-44ff-b48d-0e8f08e6bdb8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:26.085496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc975aeb-1447-44ff-b48d-0e8f08e6bdb8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:26.087056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 30/62] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:26.092395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e7f8c-e751-47b3-ae42-dcb89fb0c2d3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:26.093716 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e7f8c-e751-47b3-ae42-dcb89fb0c2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:26.094221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e7f8c-e751-47b3-ae42-dcb89fb0c2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:26.118617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-236e7f8c-e751-47b3-ae42-dcb89fb0c2d3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:26.120135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-236e7f8c-e751-47b3-ae42-dcb89fb0c2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:26.176481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:26.201001 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 33/63] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:26.616543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc975aeb-1447-44ff-b48d-0e8f08e6bdb8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:26.630117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec43286-b125-4ec4-9033-17b14a3887ea None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:26.630733 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec43286-b125-4ec4-9033-17b14a3887ea None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:26.631206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec43286-b125-4ec4-9033-17b14a3887ea None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:26.633144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 34/64] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:26.639478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf907e21-1ee7-4094-b561-59641fa5f185 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:26.640608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf907e21-1ee7-4094-b561-59641fa5f185 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:26.641095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf907e21-1ee7-4094-b561-59641fa5f185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:26.673709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cf907e21-1ee7-4094-b561-59641fa5f185 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:26.674960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cf907e21-1ee7-4094-b561-59641fa5f185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:26.769751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc975aeb-1447-44ff-b48d-0e8f08e6bdb8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:26.778527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 35/65] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:13:26.780057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f84891-b378-43f3-92de-7ee3f2dd5861 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:26.780549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f84891-b378-43f3-92de-7ee3f2dd5861 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:26.781004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f84891-b378-43f3-92de-7ee3f2dd5861 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:26.782558 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 36/66] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:26.788078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37983ad-fa4d-484d-bc5e-8227d9aba95f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:26.788573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37983ad-fa4d-484d-bc5e-8227d9aba95f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:26.789024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37983ad-fa4d-484d-bc5e-8227d9aba95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:26.797379 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:26.805419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcce30-c7a7-44c7-8aad-79d981b5bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:26.806327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcce30-c7a7-44c7-8aad-79d981b5bf8c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:26.806847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcce30-c7a7-44c7-8aad-79d981b5bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:26.807262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcce30-c7a7-44c7-8aad-79d981b5bf8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:26.810423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d37983ad-fa4d-484d-bc5e-8227d9aba95f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:26.812256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d37983ad-fa4d-484d-bc5e-8227d9aba95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:26.819219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcce30-c7a7-44c7-8aad-79d981b5bf8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-22fcce30-c7a7-44c7-8aad-79d981b5bf8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:26.824616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:26.825606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:26.826391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:26.829322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 31/67] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:26.842504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcce30-c7a7-44c7-8aad-79d981b5bf8c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:26.940217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 37/68] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:26.955747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0278e2-aa4b-47e7-8184-32aadad8fc03 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:26.956037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0278e2-aa4b-47e7-8184-32aadad8fc03 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:26.956315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0278e2-aa4b-47e7-8184-32aadad8fc03 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:26.957635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 38/69] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:26.974468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c37e4-7a24-496e-bd35-d57aafc9a10f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:26.974755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c37e4-7a24-496e-bd35-d57aafc9a10f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:26.975038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c37e4-7a24-496e-bd35-d57aafc9a10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:26.997851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcce30-c7a7-44c7-8aad-79d981b5bf8c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.012454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0b6c37e4-7a24-496e-bd35-d57aafc9a10f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:27.013658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0b6c37e4-7a24-496e-bd35-d57aafc9a10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:27.042397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5d84a-6d02-48dd-9444-32359fd6bc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.043443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5d84a-6d02-48dd-9444-32359fd6bc2d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.044211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5d84a-6d02-48dd-9444-32359fd6bc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.044811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5d84a-6d02-48dd-9444-32359fd6bc2d None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.045718 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5d84a-6d02-48dd-9444-32359fd6bc2d None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:27.046247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5d84a-6d02-48dd-9444-32359fd6bc2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:27.047772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5d84a-6d02-48dd-9444-32359fd6bc2d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:27.055514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:27.067650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 32/70] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:27.067650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f68d4-ec96-4404-8818-f5552757b368 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.067650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f68d4-ec96-4404-8818-f5552757b368 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.067650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f68d4-ec96-4404-8818-f5552757b368 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.067650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 33/71] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:27.072171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c9d26-0ae7-499e-8359-e213e309c771 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.072530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c9d26-0ae7-499e-8359-e213e309c771 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.072782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c9d26-0ae7-499e-8359-e213e309c771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.091781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-da2c9d26-0ae7-499e-8359-e213e309c771 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:27.093691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-da2c9d26-0ae7-499e-8359-e213e309c771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:27.105973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 39/72] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:27.105973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c37e4-7a24-496e-bd35-d57aafc9a10f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.157869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:27.180253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 34/73] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:27.181166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c9d26-0ae7-499e-8359-e213e309c771 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.228727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c37e4-7a24-496e-bd35-d57aafc9a10f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.272857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae65d40-f3c8-4c9a-a786-34f096cd1ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.274174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae65d40-f3c8-4c9a-a786-34f096cd1ecc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.274890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae65d40-f3c8-4c9a-a786-34f096cd1ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.275340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae65d40-f3c8-4c9a-a786-34f096cd1ecc None admin] PATH_INFO: `/v3/users/4a89a68d10724094982d478caa7d87f5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.286297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c9d26-0ae7-499e-8359-e213e309c771 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.293244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae65d40-f3c8-4c9a-a786-34f096cd1ecc None admin] RBAC: Authorizing `identity:get_user(user_id=4a89a68d10724094982d478caa7d87f5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:27.295300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae65d40-f3c8-4c9a-a786-34f096cd1ecc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:27.298362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 40/74] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/users/4a89a68d10724094982d478caa7d87f5 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:27.299292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae65d40-f3c8-4c9a-a786-34f096cd1ecc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.327674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062eb727-2633-42f0-bf38-055c184eba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.328315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062eb727-2633-42f0-bf38-055c184eba9c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.328601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062eb727-2633-42f0-bf38-055c184eba9c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.328879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062eb727-2633-42f0-bf38-055c184eba9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.334805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:27.344361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062eb727-2633-42f0-bf38-055c184eba9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-062eb727-2633-42f0-bf38-055c184eba9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:27.349369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:27.350208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:27.350966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 35/75] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:27.351578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062eb727-2633-42f0-bf38-055c184eba9c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.401696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:27.435734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae65d40-f3c8-4c9a-a786-34f096cd1ecc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.464389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062eb727-2633-42f0-bf38-055c184eba9c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.473137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e2d2d-034a-427d-8b8f-54097bad03ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.473779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e2d2d-034a-427d-8b8f-54097bad03ed None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.474094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e2d2d-034a-427d-8b8f-54097bad03ed None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.474376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e2d2d-034a-427d-8b8f-54097bad03ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.485057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e2d2d-034a-427d-8b8f-54097bad03ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-ca8e2d2d-034a-427d-8b8f-54097bad03ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:27.490285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:27.491109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:27.491840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 41/76] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:27.493004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e2d2d-034a-427d-8b8f-54097bad03ed None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.503197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db574ee3-2445-4a62-980d-18ce4c953083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.503807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db574ee3-2445-4a62-980d-18ce4c953083 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.504094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db574ee3-2445-4a62-980d-18ce4c953083 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.504372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db574ee3-2445-4a62-980d-18ce4c953083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.571458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db574ee3-2445-4a62-980d-18ce4c953083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:27.572913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db574ee3-2445-4a62-980d-18ce4c953083 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:27.582357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:27.595428 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e2d2d-034a-427d-8b8f-54097bad03ed None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.616762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:27.629139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b94fbf-8e67-42ae-85ea-9502ca289e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.629742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b94fbf-8e67-42ae-85ea-9502ca289e3a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.630061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b94fbf-8e67-42ae-85ea-9502ca289e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.630342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b94fbf-8e67-42ae-85ea-9502ca289e3a None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.630998 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b94fbf-8e67-42ae-85ea-9502ca289e3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:27.631305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b94fbf-8e67-42ae-85ea-9502ca289e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:27.632688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b94fbf-8e67-42ae-85ea-9502ca289e3a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:27.640436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 42/77] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:27.645787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b94fbf-8e67-42ae-85ea-9502ca289e3a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.674085 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 36/78] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:27.674914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db574ee3-2445-4a62-980d-18ce4c953083 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.748525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b94fbf-8e67-42ae-85ea-9502ca289e3a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.769877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db574ee3-2445-4a62-980d-18ce4c953083 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.781027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05cf16-d557-43dd-9d47-e3b4602eaabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.781655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05cf16-d557-43dd-9d47-e3b4602eaabc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.781954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05cf16-d557-43dd-9d47-e3b4602eaabc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.782237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05cf16-d557-43dd-9d47-e3b4602eaabc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.783333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05cf16-d557-43dd-9d47-e3b4602eaabc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:27.784857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05cf16-d557-43dd-9d47-e3b4602eaabc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:27.786626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 43/79] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:27.787206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05cf16-d557-43dd-9d47-e3b4602eaabc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.805775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:27.807671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7989c9-4aee-486e-9376-e26f2b067c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.808308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7989c9-4aee-486e-9376-e26f2b067c70 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.808679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7989c9-4aee-486e-9376-e26f2b067c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.808927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7989c9-4aee-486e-9376-e26f2b067c70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.816624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:27.819730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7989c9-4aee-486e-9376-e26f2b067c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3f7989c9-4aee-486e-9376-e26f2b067c70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:27.824608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:27.825477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:27.826463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 37/80] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:27.827726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7989c9-4aee-486e-9376-e26f2b067c70 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.886398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05cf16-d557-43dd-9d47-e3b4602eaabc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.919684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d9d21-c521-46b8-83c4-f7c67b0d98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.920308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d9d21-c521-46b8-83c4-f7c67b0d98c3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.920591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d9d21-c521-46b8-83c4-f7c67b0d98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.920891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d9d21-c521-46b8-83c4-f7c67b0d98c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.921556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d9d21-c521-46b8-83c4-f7c67b0d98c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:27.921861 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d9d21-c521-46b8-83c4-f7c67b0d98c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:27.923262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d9d21-c521-46b8-83c4-f7c67b0d98c3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:27.924728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7989c9-4aee-486e-9376-e26f2b067c70 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:27.931128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 44/81] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:27.931128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d9d21-c521-46b8-83c4-f7c67b0d98c3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:27.961497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142aa4d-2527-49e1-bb3b-0d6a0831a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:27.962157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142aa4d-2527-49e1-bb3b-0d6a0831a139 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:27.962440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142aa4d-2527-49e1-bb3b-0d6a0831a139 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:27.962716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142aa4d-2527-49e1-bb3b-0d6a0831a139 None admin] PATH_INFO: `/v3/users/493af8396fb3419ea552b2a7642f7a68` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:27.963853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142aa4d-2527-49e1-bb3b-0d6a0831a139 None admin] RBAC: Authorizing `identity:get_user(user_id=493af8396fb3419ea552b2a7642f7a68)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:27.965309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142aa4d-2527-49e1-bb3b-0d6a0831a139 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:27.966919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 38/82] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/users/493af8396fb3419ea552b2a7642f7a68 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:27.967648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142aa4d-2527-49e1-bb3b-0d6a0831a139 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:28.016160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:28.035695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d9d21-c521-46b8-83c4-f7c67b0d98c3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:28.037542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:28.067208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315ee90-47ff-4ed5-a8c7-192660528530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:28.067834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315ee90-47ff-4ed5-a8c7-192660528530 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:28.068118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315ee90-47ff-4ed5-a8c7-192660528530 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:28.068389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315ee90-47ff-4ed5-a8c7-192660528530 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:28.069400 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142aa4d-2527-49e1-bb3b-0d6a0831a139 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:28.069501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315ee90-47ff-4ed5-a8c7-192660528530 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:28.070938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315ee90-47ff-4ed5-a8c7-192660528530 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:28.072538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 45/83] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:27 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:28.073291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315ee90-47ff-4ed5-a8c7-192660528530 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:28.103648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb49ba-10c3-49b5-888a-9753f2c3ff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:28.104263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb49ba-10c3-49b5-888a-9753f2c3ff22 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:28.104552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb49ba-10c3-49b5-888a-9753f2c3ff22 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:28.104834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb49ba-10c3-49b5-888a-9753f2c3ff22 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:28.105501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb49ba-10c3-49b5-888a-9753f2c3ff22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:28.106948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb49ba-10c3-49b5-888a-9753f2c3ff22 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:28.134094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 39/84] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:13:27 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:28.134094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb49ba-10c3-49b5-888a-9753f2c3ff22 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:28.181215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315ee90-47ff-4ed5-a8c7-192660528530 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:28.225041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f7b6e-f3b3-4cdd-b1f0-c169588af6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:28.225719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f7b6e-f3b3-4cdd-b1f0-c169588af6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:28.226036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f7b6e-f3b3-4cdd-b1f0-c169588af6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:28.226314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f7b6e-f3b3-4cdd-b1f0-c169588af6b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:28.226999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f7b6e-f3b3-4cdd-b1f0-c169588af6b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:28.227307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f7b6e-f3b3-4cdd-b1f0-c169588af6b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:28.228732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f7b6e-f3b3-4cdd-b1f0-c169588af6b6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:28.240361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 46/85] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:28.241512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f7b6e-f3b3-4cdd-b1f0-c169588af6b6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:28.256922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:28.288549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb49ba-10c3-49b5-888a-9753f2c3ff22 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:28.314972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:28.329798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41e68b-c8a6-4596-a845-ed19b1466c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:28.330484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41e68b-c8a6-4596-a845-ed19b1466c40 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:28.330766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41e68b-c8a6-4596-a845-ed19b1466c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:28.331046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41e68b-c8a6-4596-a845-ed19b1466c40 None admin] PATH_INFO: `/v3/users/4a89a68d10724094982d478caa7d87f5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:28.332206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41e68b-c8a6-4596-a845-ed19b1466c40 None admin] RBAC: Authorizing `identity:get_user(user_id=4a89a68d10724094982d478caa7d87f5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:28.333663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41e68b-c8a6-4596-a845-ed19b1466c40 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:28.335379 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 40/86] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:28 2022] GET /identity/v3/users/4a89a68d10724094982d478caa7d87f5 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:28.340678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41e68b-c8a6-4596-a845-ed19b1466c40 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:28.358149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f7b6e-f3b3-4cdd-b1f0-c169588af6b6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:28.397890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ab713-0fcf-44d9-be0e-2c5f81b83886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:28.398850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ab713-0fcf-44d9-be0e-2c5f81b83886 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:28.399275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ab713-0fcf-44d9-be0e-2c5f81b83886 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:28.399692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ab713-0fcf-44d9-be0e-2c5f81b83886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:28.458133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41e68b-c8a6-4596-a845-ed19b1466c40 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:28.488996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ab713-0fcf-44d9-be0e-2c5f81b83886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:28.490565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ab713-0fcf-44d9-be0e-2c5f81b83886 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:28.502298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:28.503789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae5a8e-9fd7-49c2-ba23-475348c4b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:28.504412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5a8e-9fd7-49c2-ba23-475348c4b8df None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:28.504689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5a8e-9fd7-49c2-ba23-475348c4b8df None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:28.504982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5a8e-9fd7-49c2-ba23-475348c4b8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:28.506296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae5a8e-9fd7-49c2-ba23-475348c4b8df None admin] RBAC: Adding query filter params (role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=ad635390dd3d4dcf897fa77044fd970e, user.id=4a89a68d10724094982d478caa7d87f5) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:28.506608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae5a8e-9fd7-49c2-ba23-475348c4b8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:28.508061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae5a8e-9fd7-49c2-ba23-475348c4b8df None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:28.516757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 41/87] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:28 2022] GET /identity/v3/role_assignments?user.id=4a89a68d10724094982d478caa7d87f5&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=d36487377a664531a7f2e06f2f648a6f => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:28.523016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae5a8e-9fd7-49c2-ba23-475348c4b8df None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:28.618388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 47/88] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:28.619568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ab713-0fcf-44d9-be0e-2c5f81b83886 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:28.656544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:28.745175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ab713-0fcf-44d9-be0e-2c5f81b83886 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:28.749891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae5a8e-9fd7-49c2-ba23-475348c4b8df None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:28.758327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:28.788971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edda32-0fbd-4c95-9ce4-5e7a9a11f616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:28.790260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edda32-0fbd-4c95-9ce4-5e7a9a11f616 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:28.790742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edda32-0fbd-4c95-9ce4-5e7a9a11f616 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:28.791505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edda32-0fbd-4c95-9ce4-5e7a9a11f616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:28.846691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8293f2-21c8-408a-87b7-295e39953e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:28.847381 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8293f2-21c8-408a-87b7-295e39953e21 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:28.847668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8293f2-21c8-408a-87b7-295e39953e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:28.847950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8293f2-21c8-408a-87b7-295e39953e21 None admin] PATH_INFO: `/v3/users/4a89a68d10724094982d478caa7d87f5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:28.854645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8293f2-21c8-408a-87b7-295e39953e21 None admin] RBAC: Authorizing `identity:get_user(user_id=4a89a68d10724094982d478caa7d87f5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:28.856108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8293f2-21c8-408a-87b7-295e39953e21 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:28.857883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 42/89] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:28 2022] GET /identity/v3/users/4a89a68d10724094982d478caa7d87f5 => generated 275 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:28.858580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8293f2-21c8-408a-87b7-295e39953e21 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:28.872513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edda32-0fbd-4c95-9ce4-5e7a9a11f616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:28.874455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edda32-0fbd-4c95-9ce4-5e7a9a11f616 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:28.987784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 48/90] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:28.989340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edda32-0fbd-4c95-9ce4-5e7a9a11f616 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:29.004188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:29.109651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8293f2-21c8-408a-87b7-295e39953e21 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:29.135040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edda32-0fbd-4c95-9ce4-5e7a9a11f616 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:29.135687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:29.153299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b148f92-4616-42f0-aca5-0e51ac4fb3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:29.156212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b148f92-4616-42f0-aca5-0e51ac4fb3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:29.158016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b148f92-4616-42f0-aca5-0e51ac4fb3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:29.159143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b148f92-4616-42f0-aca5-0e51ac4fb3f7 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:29.162035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b148f92-4616-42f0-aca5-0e51ac4fb3f7 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:29.164576 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b148f92-4616-42f0-aca5-0e51ac4fb3f7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:29.169963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 43/91] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:28 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:29.169963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b148f92-4616-42f0-aca5-0e51ac4fb3f7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:29.205478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184d38b-af6e-4a77-a82f-d2fa95ae73c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:29.206206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184d38b-af6e-4a77-a82f-d2fa95ae73c6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:29.206413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184d38b-af6e-4a77-a82f-d2fa95ae73c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:29.206688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184d38b-af6e-4a77-a82f-d2fa95ae73c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:29.338360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b148f92-4616-42f0-aca5-0e51ac4fb3f7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:29.361129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184d38b-af6e-4a77-a82f-d2fa95ae73c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:29.362621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184d38b-af6e-4a77-a82f-d2fa95ae73c6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:29.394678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:29.403065 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b22b0c-8695-48b3-a385-282d5e694ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:29.403758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22b0c-8695-48b3-a385-282d5e694ada None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:29.404042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22b0c-8695-48b3-a385-282d5e694ada None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:29.404313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22b0c-8695-48b3-a385-282d5e694ada None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:29.406442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b22b0c-8695-48b3-a385-282d5e694ada None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:29.407900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b22b0c-8695-48b3-a385-282d5e694ada None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:29.411317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 44/92] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:29 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:29.412097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b22b0c-8695-48b3-a385-282d5e694ada None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:29.533832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:29.553718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 49/93] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:29.554875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184d38b-af6e-4a77-a82f-d2fa95ae73c6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:29.584864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b22b0c-8695-48b3-a385-282d5e694ada None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:29.628022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c5e04-0d2c-4705-bc1f-efa8d9bf60e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:29.629844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c5e04-0d2c-4705-bc1f-efa8d9bf60e4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:29.630735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c5e04-0d2c-4705-bc1f-efa8d9bf60e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:29.631020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c5e04-0d2c-4705-bc1f-efa8d9bf60e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:29.631755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c5e04-0d2c-4705-bc1f-efa8d9bf60e4 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=ad635390dd3d4dcf897fa77044fd970e, user.id=493af8396fb3419ea552b2a7642f7a68) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:29.632069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c5e04-0d2c-4705-bc1f-efa8d9bf60e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:29.633521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c5e04-0d2c-4705-bc1f-efa8d9bf60e4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:29.647605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 45/94] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:29 2022] GET /identity/v3/role_assignments?user.id=493af8396fb3419ea552b2a7642f7a68&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:29.648453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c5e04-0d2c-4705-bc1f-efa8d9bf60e4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:29.780914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184d38b-af6e-4a77-a82f-d2fa95ae73c6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:29.794147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c5e04-0d2c-4705-bc1f-efa8d9bf60e4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:29.805504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:29.823163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:29.838837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c60fd-0162-4736-aaf2-21869c8e0ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:29.839493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c60fd-0162-4736-aaf2-21869c8e0ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:29.839781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c60fd-0162-4736-aaf2-21869c8e0ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:29.840062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c60fd-0162-4736-aaf2-21869c8e0ab5 None admin] PATH_INFO: `/v3/projects/169129b4cc1b4357ae8cdd36a8665553` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:29.842047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c60fd-0162-4736-aaf2-21869c8e0ab5 None admin] RBAC: Authorizing `identity:get_project(project_id=169129b4cc1b4357ae8cdd36a8665553)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:29.844114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c60fd-0162-4736-aaf2-21869c8e0ab5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:29.845720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 46/95] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:29 2022] GET /identity/v3/projects/169129b4cc1b4357ae8cdd36a8665553 => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:29.852740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c60fd-0162-4736-aaf2-21869c8e0ab5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:29.861748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241389c3-bc5e-4f8e-b126-49d9c6b4125f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:29.862399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241389c3-bc5e-4f8e-b126-49d9c6b4125f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:29.862685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241389c3-bc5e-4f8e-b126-49d9c6b4125f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:29.869110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241389c3-bc5e-4f8e-b126-49d9c6b4125f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:29.870030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241389c3-bc5e-4f8e-b126-49d9c6b4125f None admin] RBAC: Adding query filter params (role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=ad635390dd3d4dcf897fa77044fd970e, user.id=4a89a68d10724094982d478caa7d87f5) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:29.870295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241389c3-bc5e-4f8e-b126-49d9c6b4125f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:29.871874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241389c3-bc5e-4f8e-b126-49d9c6b4125f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:29.883056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 50/96] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:29 2022] GET /identity/v3/role_assignments?user.id=4a89a68d10724094982d478caa7d87f5&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 259 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:29.987414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c60fd-0162-4736-aaf2-21869c8e0ab5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:30.025270 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f799a37-a583-4418-9e6e-999f03d65050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:30.025882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f799a37-a583-4418-9e6e-999f03d65050 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:30.026200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f799a37-a583-4418-9e6e-999f03d65050 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:30.026482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f799a37-a583-4418-9e6e-999f03d65050 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:30.027229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f799a37-a583-4418-9e6e-999f03d65050 None admin] RBAC: Adding query filter params (role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=169129b4cc1b4357ae8cdd36a8665553, user.id=4a89a68d10724094982d478caa7d87f5) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:30.027542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f799a37-a583-4418-9e6e-999f03d65050 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:30.029043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f799a37-a583-4418-9e6e-999f03d65050 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:30.036155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 47/97] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:29 2022] GET /identity/v3/role_assignments?user.id=4a89a68d10724094982d478caa7d87f5&scope.project.id=169129b4cc1b4357ae8cdd36a8665553&role.id=d36487377a664531a7f2e06f2f648a6f => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:30.787478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4eb4b-1e83-454e-9e70-e89bd12e22d0 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:30.787977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4eb4b-1e83-454e-9e70-e89bd12e22d0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:30.788079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4eb4b-1e83-454e-9e70-e89bd12e22d0 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:30.789569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 51/98] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:30.795249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a4f5f-b798-4bbd-bdfc-7b66cbb12ca0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:30.796146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a4f5f-b798-4bbd-bdfc-7b66cbb12ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:30.796625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a4f5f-b798-4bbd-bdfc-7b66cbb12ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:30.817113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1d5a4f5f-b798-4bbd-bdfc-7b66cbb12ca0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:30.818694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1d5a4f5f-b798-4bbd-bdfc-7b66cbb12ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:30.874981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:30.894434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 48/99] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:31.199647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46093205-5e60-4827-9bf4-f58189b196f4 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:31.200284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46093205-5e60-4827-9bf4-f58189b196f4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:31.200806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46093205-5e60-4827-9bf4-f58189b196f4 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:31.202943 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 52/100] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:31.208338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f46303-35bc-44b6-8531-1ec82e45d24a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:31.208922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f46303-35bc-44b6-8531-1ec82e45d24a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:31.209463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f46303-35bc-44b6-8531-1ec82e45d24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:31.234047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-25f46303-35bc-44b6-8531-1ec82e45d24a None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:31.235660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-25f46303-35bc-44b6-8531-1ec82e45d24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:31.321448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 49/101] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:31.332592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbde27-646b-4cc2-a52d-baacd5e123fd None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:31.333151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbde27-646b-4cc2-a52d-baacd5e123fd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:31.333631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbde27-646b-4cc2-a52d-baacd5e123fd None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:31.335198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 53/102] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:31.340610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a5705-cfc8-4904-b747-e6fd70ed532e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:31.341123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a5705-cfc8-4904-b747-e6fd70ed532e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:31.341618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a5705-cfc8-4904-b747-e6fd70ed532e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:31.361711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-700a5705-cfc8-4904-b747-e6fd70ed532e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:31.363502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-700a5705-cfc8-4904-b747-e6fd70ed532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:31.435595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbde27-646b-4cc2-a52d-baacd5e123fd None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:31.439202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 50/103] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:13:31.563893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbde27-646b-4cc2-a52d-baacd5e123fd None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:31.609336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be8b09-0468-4372-beeb-7cde8b7b2e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:31.610371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be8b09-0468-4372-beeb-7cde8b7b2e14 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:31.610847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be8b09-0468-4372-beeb-7cde8b7b2e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:31.611337 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be8b09-0468-4372-beeb-7cde8b7b2e14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:31.612698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be8b09-0468-4372-beeb-7cde8b7b2e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:31.614684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be8b09-0468-4372-beeb-7cde8b7b2e14 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:31.620183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 54/104] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:31.622598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a5705-cfc8-4904-b747-e6fd70ed532e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:31.680909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:31.745953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a5705-cfc8-4904-b747-e6fd70ed532e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:31.795700 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af9b0b-6f43-4771-ae4a-4f644fbfa24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:31.796414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af9b0b-6f43-4771-ae4a-4f644fbfa24a None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:31.796609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af9b0b-6f43-4771-ae4a-4f644fbfa24a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:31.796882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af9b0b-6f43-4771-ae4a-4f644fbfa24a None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:31.797546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af9b0b-6f43-4771-ae4a-4f644fbfa24a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:31.799022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af9b0b-6f43-4771-ae4a-4f644fbfa24a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:31.815171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be8b09-0468-4372-beeb-7cde8b7b2e14 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:31.855585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 51/105] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:13:31 2022] POST /identity/v3/projects => generated 302 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:31.878587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:31.950047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be8b09-0468-4372-beeb-7cde8b7b2e14 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:31.990232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942cd81-d382-4e91-8640-d6ea8358adb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:31.990956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942cd81-d382-4e91-8640-d6ea8358adb3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:31.991238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942cd81-d382-4e91-8640-d6ea8358adb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:31.991561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942cd81-d382-4e91-8640-d6ea8358adb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:32.010562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942cd81-d382-4e91-8640-d6ea8358adb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0942cd81-d382-4e91-8640-d6ea8358adb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:32.016533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:32.017408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:32.018210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 55/106] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:13:32.024139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af9b0b-6f43-4771-ae4a-4f644fbfa24a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:32.059926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942cd81-d382-4e91-8640-d6ea8358adb3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:32.191606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:32.196229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af9b0b-6f43-4771-ae4a-4f644fbfa24a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:32.216776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942cd81-d382-4e91-8640-d6ea8358adb3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:32.220014 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:32.261609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28bb29-d4cc-4f17-84d5-00be869be74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:32.262549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28bb29-d4cc-4f17-84d5-00be869be74b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:32.262882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28bb29-d4cc-4f17-84d5-00be869be74b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:32.263162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28bb29-d4cc-4f17-84d5-00be869be74b None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:32.263840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28bb29-d4cc-4f17-84d5-00be869be74b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:32.264149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28bb29-d4cc-4f17-84d5-00be869be74b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:32.265790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28bb29-d4cc-4f17-84d5-00be869be74b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:32.267390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a2184-08d1-4cdb-8ba5-fd51f33b5f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:32.269189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2184-08d1-4cdb-8ba5-fd51f33b5f54 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:32.269691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2184-08d1-4cdb-8ba5-fd51f33b5f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:32.270229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2184-08d1-4cdb-8ba5-fd51f33b5f54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:32.281974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 52/107] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:32.291132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28bb29-d4cc-4f17-84d5-00be869be74b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:32.292297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a2184-08d1-4cdb-8ba5-fd51f33b5f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-904a2184-08d1-4cdb-8ba5-fd51f33b5f54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:32.300877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:32.301717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:32.302519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:32.305550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 56/108] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:13:32.311820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a2184-08d1-4cdb-8ba5-fd51f33b5f54 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:32.481149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28bb29-d4cc-4f17-84d5-00be869be74b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:32.496818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a2184-08d1-4cdb-8ba5-fd51f33b5f54 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:32.548992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ff641-5253-4c9b-9bfb-04d068cc6e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:32.550102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ff641-5253-4c9b-9bfb-04d068cc6e99 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:32.550579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ff641-5253-4c9b-9bfb-04d068cc6e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:32.551047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ff641-5253-4c9b-9bfb-04d068cc6e99 None admin] PATH_INFO: `/v3/users/1a2bf170dbcc4f11a4d52fddbe668ebd` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:32.564727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6bfd6-2c97-43e1-a84b-40d29deaf381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:32.565771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6bfd6-2c97-43e1-a84b-40d29deaf381 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:32.566282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6bfd6-2c97-43e1-a84b-40d29deaf381 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:32.566748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6bfd6-2c97-43e1-a84b-40d29deaf381 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:32.567609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6bfd6-2c97-43e1-a84b-40d29deaf381 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:32.568103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6bfd6-2c97-43e1-a84b-40d29deaf381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:32.570470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6bfd6-2c97-43e1-a84b-40d29deaf381 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:32.583613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ff641-5253-4c9b-9bfb-04d068cc6e99 None admin] RBAC: Authorizing `identity:get_user(user_id=1a2bf170dbcc4f11a4d52fddbe668ebd)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:32.586594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ff641-5253-4c9b-9bfb-04d068cc6e99 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:32.586835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 53/109] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:32 2022] GET /identity/v3/users/1a2bf170dbcc4f11a4d52fddbe668ebd => generated 289 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:32.610210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 57/110] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:32.610995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ff641-5253-4c9b-9bfb-04d068cc6e99 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:32.619456 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6bfd6-2c97-43e1-a84b-40d29deaf381 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:32.635196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:32.658042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:32.770177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ff641-5253-4c9b-9bfb-04d068cc6e99 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:32.823973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a785ebb-1d26-4ffc-9347-13178d31e0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:32.825377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a785ebb-1d26-4ffc-9347-13178d31e0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:32.825853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a785ebb-1d26-4ffc-9347-13178d31e0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:32.826356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a785ebb-1d26-4ffc-9347-13178d31e0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:32.853208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6bfd6-2c97-43e1-a84b-40d29deaf381 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:32.937328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b38dff-72c6-4a28-a6ff-14957f463ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:32.938380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b38dff-72c6-4a28-a6ff-14957f463ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:32.938808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b38dff-72c6-4a28-a6ff-14957f463ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:32.939400 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b38dff-72c6-4a28-a6ff-14957f463ac5 None admin] PATH_INFO: `/v3/users/4a89a68d10724094982d478caa7d87f5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:32.941079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b38dff-72c6-4a28-a6ff-14957f463ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=4a89a68d10724094982d478caa7d87f5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:32.943047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b38dff-72c6-4a28-a6ff-14957f463ac5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:32.949959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 58/111] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:32 2022] GET /identity/v3/users/4a89a68d10724094982d478caa7d87f5 => generated 275 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:32.952794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b38dff-72c6-4a28-a6ff-14957f463ac5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:32.975238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a785ebb-1d26-4ffc-9347-13178d31e0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:32.977046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a785ebb-1d26-4ffc-9347-13178d31e0f4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:33.013211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:33.125031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:33.140509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b38dff-72c6-4a28-a6ff-14957f463ac5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:33.164185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 54/112] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:33.165681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddea4a-cab0-48ff-8df8-e07878238dbd None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.166903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddea4a-cab0-48ff-8df8-e07878238dbd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.167567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddea4a-cab0-48ff-8df8-e07878238dbd None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.170310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 55/113] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:33.172497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddea4a-cab0-48ff-8df8-e07878238dbd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:33.210710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434f475-577c-4528-8b62-5f6ac8824eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:33.211930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434f475-577c-4528-8b62-5f6ac8824eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.212608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434f475-577c-4528-8b62-5f6ac8824eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.218115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434f475-577c-4528-8b62-5f6ac8824eb3 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.221452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434f475-577c-4528-8b62-5f6ac8824eb3 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:33.223284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434f475-577c-4528-8b62-5f6ac8824eb3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:33.230461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 59/114] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:32 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:33.232522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a70a1-23b4-4759-a212-c974ee7f8921 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.234240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a70a1-23b4-4759-a212-c974ee7f8921 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.234965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a70a1-23b4-4759-a212-c974ee7f8921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.266990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7a0a70a1-23b4-4759-a212-c974ee7f8921 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:33.268219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7a0a70a1-23b4-4759-a212-c974ee7f8921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:33.345083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 60/115] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:33.345959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a70a1-23b4-4759-a212-c974ee7f8921 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:33.397957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddea4a-cab0-48ff-8df8-e07878238dbd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:33.429458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:33.462340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaf2e1-c994-4857-858b-6ecb954ca74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:33.462980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf2e1-c994-4857-858b-6ecb954ca74d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.463257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf2e1-c994-4857-858b-6ecb954ca74d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.463531 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf2e1-c994-4857-858b-6ecb954ca74d None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.466404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaf2e1-c994-4857-858b-6ecb954ca74d None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:33.468504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaf2e1-c994-4857-858b-6ecb954ca74d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:33.476709 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 56/116] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:33 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:33.477925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d725c2-c004-4f6c-97d4-912c6064a5ff None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.478230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d725c2-c004-4f6c-97d4-912c6064a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.478507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d725c2-c004-4f6c-97d4-912c6064a5ff None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.479719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 57/117] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:33.484030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d725c2-c004-4f6c-97d4-912c6064a5ff None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:33.499679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:33.511746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a70a1-23b4-4759-a212-c974ee7f8921 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:33.597319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049e536-ab59-4982-b341-7de2761ab6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:33.598162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049e536-ab59-4982-b341-7de2761ab6cf None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.598441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049e536-ab59-4982-b341-7de2761ab6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.598718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049e536-ab59-4982-b341-7de2761ab6cf None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e/users/4a89a68d10724094982d478caa7d87f5/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.609296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d725c2-c004-4f6c-97d4-912c6064a5ff None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:33.610722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049e536-ab59-4982-b341-7de2761ab6cf None admin] RBAC: Authorizing `identity:create_grant(project_id=ad635390dd3d4dcf897fa77044fd970e, user_id=4a89a68d10724094982d478caa7d87f5, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:33.612198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049e536-ab59-4982-b341-7de2761ab6cf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:33.644204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f11d19-5ef0-4e4a-b2f4-7f83493030a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:33.644404 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 61/118] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:13:33 2022] PUT /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e/users/4a89a68d10724094982d478caa7d87f5/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:13:33.645096 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f11d19-5ef0-4e4a-b2f4-7f83493030a6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.645383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f11d19-5ef0-4e4a-b2f4-7f83493030a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.645624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ce9a3-fa6e-4083-b9fd-cd0b6c1c0347 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.645729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f11d19-5ef0-4e4a-b2f4-7f83493030a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.645911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ce9a3-fa6e-4083-b9fd-cd0b6c1c0347 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.646210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ce9a3-fa6e-4083-b9fd-cd0b6c1c0347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.646396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f11d19-5ef0-4e4a-b2f4-7f83493030a6 None admin] RBAC: Adding query filter params (role.id=439053f3a99d43f5abe2fb7dec39df9d, scope.project.id=ad635390dd3d4dcf897fa77044fd970e, user.id=1a2bf170dbcc4f11a4d52fddbe668ebd) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:33.646741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f11d19-5ef0-4e4a-b2f4-7f83493030a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:33.648185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f11d19-5ef0-4e4a-b2f4-7f83493030a6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:33.654106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 58/119] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:33 2022] GET /identity/v3/role_assignments?user.id=1a2bf170dbcc4f11a4d52fddbe668ebd&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=439053f3a99d43f5abe2fb7dec39df9d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:33.661273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46a22c-fcdd-46fb-a66c-79ac61762021 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.661545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46a22c-fcdd-46fb-a66c-79ac61762021 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.661820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46a22c-fcdd-46fb-a66c-79ac61762021 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.663214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 59/120] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:33.664005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46a22c-fcdd-46fb-a66c-79ac61762021 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:33.693455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e16ce9a3-fa6e-4083-b9fd-cd0b6c1c0347 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:33.694730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e16ce9a3-fa6e-4083-b9fd-cd0b6c1c0347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:33.739454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:33.808280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46a22c-fcdd-46fb-a66c-79ac61762021 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:33.848688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f0729-693f-428e-a1fd-7176ab989cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:33.849398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f0729-693f-428e-a1fd-7176ab989cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.849685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f0729-693f-428e-a1fd-7176ab989cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.850594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f0729-693f-428e-a1fd-7176ab989cd1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.878487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f0729-693f-428e-a1fd-7176ab989cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-484f0729-693f-428e-a1fd-7176ab989cd1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:33.887604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:33.888529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:33.889837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 60/121] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:13:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:33.894413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ca493-1351-4490-8fe4-5b0db9241339 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:33.894628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ca493-1351-4490-8fe4-5b0db9241339 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:33.894910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ca493-1351-4490-8fe4-5b0db9241339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:33.928483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 62/122] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:33.929976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ce9a3-fa6e-4083-b9fd-cd0b6c1c0347 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:33.935697 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4b8ca493-1351-4490-8fe4-5b0db9241339 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:33.936919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4b8ca493-1351-4490-8fe4-5b0db9241339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:34.069036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 61/123] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:34.103651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:34.145669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ce9a3-fa6e-4083-b9fd-cd0b6c1c0347 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:34.231710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b7776-b145-4114-92f3-a6deb7d31a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:34.232257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b7776-b145-4114-92f3-a6deb7d31a4c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:34.232537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b7776-b145-4114-92f3-a6deb7d31a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:34.232807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b7776-b145-4114-92f3-a6deb7d31a4c None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:34.233516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b7776-b145-4114-92f3-a6deb7d31a4c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:34.233817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b7776-b145-4114-92f3-a6deb7d31a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:34.235237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b7776-b145-4114-92f3-a6deb7d31a4c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:34.257377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 63/124] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:13:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:34.262779 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ca493-1351-4490-8fe4-5b0db9241339 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:34.331052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:34.396552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b7776-b145-4114-92f3-a6deb7d31a4c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:34.400390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ca493-1351-4490-8fe4-5b0db9241339 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:34.435607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009713a5-cba2-4b33-a7f9-7bc6ff1c77ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:34.436779 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009713a5-cba2-4b33-a7f9-7bc6ff1c77ee None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:34.437061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009713a5-cba2-4b33-a7f9-7bc6ff1c77ee None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:34.437333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009713a5-cba2-4b33-a7f9-7bc6ff1c77ee None admin] PATH_INFO: `/v3/users/4a89a68d10724094982d478caa7d87f5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:34.439042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009713a5-cba2-4b33-a7f9-7bc6ff1c77ee None admin] RBAC: Authorizing `identity:get_user(user_id=4a89a68d10724094982d478caa7d87f5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:34.440943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009713a5-cba2-4b33-a7f9-7bc6ff1c77ee None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:34.443242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 62/125] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:34 2022] GET /identity/v3/users/4a89a68d10724094982d478caa7d87f5 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:34.448753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009713a5-cba2-4b33-a7f9-7bc6ff1c77ee None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:34.568397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009713a5-cba2-4b33-a7f9-7bc6ff1c77ee None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:34.578869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:34.583196 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b7776-b145-4114-92f3-a6deb7d31a4c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:34.615511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bc1c5-ba26-4444-b7ee-689572714254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:34.621952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bc1c5-ba26-4444-b7ee-689572714254 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:34.621952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bc1c5-ba26-4444-b7ee-689572714254 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:34.621952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bc1c5-ba26-4444-b7ee-689572714254 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:34.621952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bc1c5-ba26-4444-b7ee-689572714254 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:34.621952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bc1c5-ba26-4444-b7ee-689572714254 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:34.626022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 63/126] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:34 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:34.627076 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bc1c5-ba26-4444-b7ee-689572714254 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:34.663482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fc6c2-a3a6-42de-94f1-ba801b607f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:34.664122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fc6c2-a3a6-42de-94f1-ba801b607f51 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:34.664400 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fc6c2-a3a6-42de-94f1-ba801b607f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:34.664724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fc6c2-a3a6-42de-94f1-ba801b607f51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:34.684615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fc6c2-a3a6-42de-94f1-ba801b607f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c14fc6c2-a3a6-42de-94f1-ba801b607f51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:34.699134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:34.700061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:34.700799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 64/127] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:13:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:13:34.701410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fc6c2-a3a6-42de-94f1-ba801b607f51 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:34.721022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:34.816616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bc1c5-ba26-4444-b7ee-689572714254 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:34.870309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ed1f1-d88a-4485-ae71-02bf5c785cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:34.871213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ed1f1-d88a-4485-ae71-02bf5c785cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:34.871504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ed1f1-d88a-4485-ae71-02bf5c785cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:34.872194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ed1f1-d88a-4485-ae71-02bf5c785cb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:34.875358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fc6c2-a3a6-42de-94f1-ba801b607f51 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:34.884406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ed1f1-d88a-4485-ae71-02bf5c785cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-9c9ed1f1-d88a-4485-ae71-02bf5c785cb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:34.890427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:34.891274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:34.892511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 64/128] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:34.893419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ed1f1-d88a-4485-ae71-02bf5c785cb8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:34.910995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5c3f8-ab87-4c8d-b0a0-ca06c5564166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:34.911706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5c3f8-ab87-4c8d-b0a0-ca06c5564166 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:34.911992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5c3f8-ab87-4c8d-b0a0-ca06c5564166 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:34.912265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5c3f8-ab87-4c8d-b0a0-ca06c5564166 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e/users/4a89a68d10724094982d478caa7d87f5/roles/9578cd8bc0a8498f99757fd0111e48e8` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:34.914489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5c3f8-ab87-4c8d-b0a0-ca06c5564166 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad635390dd3d4dcf897fa77044fd970e, user_id=4a89a68d10724094982d478caa7d87f5, role_id=9578cd8bc0a8498f99757fd0111e48e8)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:34.915955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5c3f8-ab87-4c8d-b0a0-ca06c5564166 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:34.934807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 65/129] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:13:34 2022] PUT /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e/users/4a89a68d10724094982d478caa7d87f5/roles/9578cd8bc0a8498f99757fd0111e48e8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:13:34.936087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeedc88-3ed3-487c-92d4-b1de8ea10eeb None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:34.937300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeedc88-3ed3-487c-92d4-b1de8ea10eeb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:34.937470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeedc88-3ed3-487c-92d4-b1de8ea10eeb None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:34.938313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:34.938890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 66/130] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:34.940534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeedc88-3ed3-487c-92d4-b1de8ea10eeb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:35.053669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:35.074706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ed1f1-d88a-4485-ae71-02bf5c785cb8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:35.111765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeedc88-3ed3-487c-92d4-b1de8ea10eeb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:35.134397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b0985-f7df-42c0-920c-e4b5d4952d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:35.134936 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b0985-f7df-42c0-920c-e4b5d4952d23 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:35.135217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b0985-f7df-42c0-920c-e4b5d4952d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:35.135489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b0985-f7df-42c0-920c-e4b5d4952d23 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:35.136776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b0985-f7df-42c0-920c-e4b5d4952d23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:35.137141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b0985-f7df-42c0-920c-e4b5d4952d23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:35.138884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b0985-f7df-42c0-920c-e4b5d4952d23 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:35.153978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 65/131] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:35.153978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5bc1-d4d3-4c78-804b-e5837fb17f67 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:35.153978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5bc1-d4d3-4c78-804b-e5837fb17f67 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:35.153978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5bc1-d4d3-4c78-804b-e5837fb17f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:35.167165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068453d-8cc5-46c8-bb42-4a55a65f41c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:35.168095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068453d-8cc5-46c8-bb42-4a55a65f41c2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:35.168586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068453d-8cc5-46c8-bb42-4a55a65f41c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:35.169049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068453d-8cc5-46c8-bb42-4a55a65f41c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:35.169899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068453d-8cc5-46c8-bb42-4a55a65f41c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:35.170419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068453d-8cc5-46c8-bb42-4a55a65f41c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:35.172614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068453d-8cc5-46c8-bb42-4a55a65f41c2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:35.174044 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-dabe5bc1-d4d3-4c78-804b-e5837fb17f67 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:35.175554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dabe5bc1-d4d3-4c78-804b-e5837fb17f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:35.181855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 67/132] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:35.183919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068453d-8cc5-46c8-bb42-4a55a65f41c2 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:35.258599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 66/133] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:35.260334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe5bc1-d4d3-4c78-804b-e5837fb17f67 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:35.292477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:35.311347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068453d-8cc5-46c8-bb42-4a55a65f41c2 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:35.341921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:35.360263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a910363-246d-49c0-9b79-dd1cac521eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:35.361238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a910363-246d-49c0-9b79-dd1cac521eee None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:35.362763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a910363-246d-49c0-9b79-dd1cac521eee None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:35.363371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a910363-246d-49c0-9b79-dd1cac521eee None admin] PATH_INFO: `/v3/users/493af8396fb3419ea552b2a7642f7a68` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:35.364846 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a910363-246d-49c0-9b79-dd1cac521eee None admin] RBAC: Authorizing `identity:get_user(user_id=493af8396fb3419ea552b2a7642f7a68)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:35.367023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a910363-246d-49c0-9b79-dd1cac521eee None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:35.368670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 68/134] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:35 2022] GET /identity/v3/users/493af8396fb3419ea552b2a7642f7a68 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:35.374453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a910363-246d-49c0-9b79-dd1cac521eee None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:35.386372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe5bc1-d4d3-4c78-804b-e5837fb17f67 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:35.452971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc59e9-1ece-483b-b173-d7815855fbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:35.454326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc59e9-1ece-483b-b173-d7815855fbaf None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:35.454608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc59e9-1ece-483b-b173-d7815855fbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:35.454878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc59e9-1ece-483b-b173-d7815855fbaf None admin] PATH_INFO: `/v3/users/4a89a68d10724094982d478caa7d87f5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:35.458399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc59e9-1ece-483b-b173-d7815855fbaf None admin] RBAC: Authorizing `identity:get_user(user_id=4a89a68d10724094982d478caa7d87f5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:35.459844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc59e9-1ece-483b-b173-d7815855fbaf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:35.463552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 67/135] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:35 2022] GET /identity/v3/users/4a89a68d10724094982d478caa7d87f5 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:35.469070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc59e9-1ece-483b-b173-d7815855fbaf None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:35.571174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a910363-246d-49c0-9b79-dd1cac521eee None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:35.584798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:35.592305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc59e9-1ece-483b-b173-d7815855fbaf None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:35.635070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb52b3a-0c3f-4994-9394-e63dde6db901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:35.636134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb52b3a-0c3f-4994-9394-e63dde6db901 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:35.636678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb52b3a-0c3f-4994-9394-e63dde6db901 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:35.637140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb52b3a-0c3f-4994-9394-e63dde6db901 None admin] PATH_INFO: `/v3/projects/169129b4cc1b4357ae8cdd36a8665553` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:35.638477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb52b3a-0c3f-4994-9394-e63dde6db901 None admin] RBAC: Authorizing `identity:get_project(project_id=169129b4cc1b4357ae8cdd36a8665553)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:35.640269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb52b3a-0c3f-4994-9394-e63dde6db901 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:35.642208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 68/136] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:35 2022] GET /identity/v3/projects/169129b4cc1b4357ae8cdd36a8665553 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:35.643690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582ad67-a83c-43b5-8c68-a50579834d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:35.644338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582ad67-a83c-43b5-8c68-a50579834d06 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:35.644621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582ad67-a83c-43b5-8c68-a50579834d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:35.644897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582ad67-a83c-43b5-8c68-a50579834d06 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:35.649178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582ad67-a83c-43b5-8c68-a50579834d06 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:35.649178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582ad67-a83c-43b5-8c68-a50579834d06 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:35.650287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb52b3a-0c3f-4994-9394-e63dde6db901 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:35.653967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 69/137] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:35 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:35.669963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582ad67-a83c-43b5-8c68-a50579834d06 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:35.798985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb52b3a-0c3f-4994-9394-e63dde6db901 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:35.807371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:35.844202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0181f0e-cf79-40a9-bae6-0c86fa8fae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:35.845365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0181f0e-cf79-40a9-bae6-0c86fa8fae4c None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:35.845864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0181f0e-cf79-40a9-bae6-0c86fa8fae4c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:35.846462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0181f0e-cf79-40a9-bae6-0c86fa8fae4c None admin] PATH_INFO: `/v3/projects/169129b4cc1b4357ae8cdd36a8665553/users/4a89a68d10724094982d478caa7d87f5/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:35.848581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0181f0e-cf79-40a9-bae6-0c86fa8fae4c None admin] RBAC: Authorizing `identity:create_grant(project_id=169129b4cc1b4357ae8cdd36a8665553, user_id=4a89a68d10724094982d478caa7d87f5, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:35.850469 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0181f0e-cf79-40a9-bae6-0c86fa8fae4c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:35.877456 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 69/138] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:13:35 2022] PUT /identity/v3/projects/169129b4cc1b4357ae8cdd36a8665553/users/4a89a68d10724094982d478caa7d87f5/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:13:35.894128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582ad67-a83c-43b5-8c68-a50579834d06 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:35.903227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0181f0e-cf79-40a9-bae6-0c86fa8fae4c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:35.955985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:35.956331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef2102-2860-45c6-97a7-df5f74379585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:35.956897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef2102-2860-45c6-97a7-df5f74379585 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:35.957187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef2102-2860-45c6-97a7-df5f74379585 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:35.957469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef2102-2860-45c6-97a7-df5f74379585 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e/users/493af8396fb3419ea552b2a7642f7a68/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:35.961815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef2102-2860-45c6-97a7-df5f74379585 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad635390dd3d4dcf897fa77044fd970e, user_id=493af8396fb3419ea552b2a7642f7a68, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:35.963381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef2102-2860-45c6-97a7-df5f74379585 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:35.990623 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 70/139] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:13:35 2022] PUT /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e/users/493af8396fb3419ea552b2a7642f7a68/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:13:36.065811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0181f0e-cf79-40a9-bae6-0c86fa8fae4c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:36.108910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb1f49-01ca-4709-b432-61a0626d59b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:36.109563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb1f49-01ca-4709-b432-61a0626d59b2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:36.109846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb1f49-01ca-4709-b432-61a0626d59b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:36.110149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb1f49-01ca-4709-b432-61a0626d59b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:36.111382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb1f49-01ca-4709-b432-61a0626d59b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:36.113122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb1f49-01ca-4709-b432-61a0626d59b2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:36.114837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 70/140] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:36.120747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef2102-2860-45c6-97a7-df5f74379585 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:36.260898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef2102-2860-45c6-97a7-df5f74379585 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:36.288434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:36.309776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3a64b-af0a-4b2d-8698-b330be361f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:36.310437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3a64b-af0a-4b2d-8698-b330be361f41 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:36.310720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3a64b-af0a-4b2d-8698-b330be361f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:36.311000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3a64b-af0a-4b2d-8698-b330be361f41 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:36.311786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3a64b-af0a-4b2d-8698-b330be361f41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:36.313257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3a64b-af0a-4b2d-8698-b330be361f41 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:36.349509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 71/141] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:13:36 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:36.690494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa248cd-a061-4186-9c5a-ccf8a2b4256f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:36.690831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa248cd-a061-4186-9c5a-ccf8a2b4256f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:36.691061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa248cd-a061-4186-9c5a-ccf8a2b4256f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:36.692538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 71/142] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:36.698274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105ae28-c801-48fe-940d-cbb50e544e0d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:36.698567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105ae28-c801-48fe-940d-cbb50e544e0d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:36.698850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105ae28-c801-48fe-940d-cbb50e544e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:36.719816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1105ae28-c801-48fe-940d-cbb50e544e0d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:36.721020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1105ae28-c801-48fe-940d-cbb50e544e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:36.805768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 72/143] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:36.893423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8f175-af89-4371-ab4f-079671aa2747 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:36.894378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8f175-af89-4371-ab4f-079671aa2747 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:36.894876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8f175-af89-4371-ab4f-079671aa2747 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:36.896706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 72/144] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:36.901740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f368a-04a7-41a3-bc8f-e247ea1e4c38 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:36.902062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f368a-04a7-41a3-bc8f-e247ea1e4c38 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:36.902343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f368a-04a7-41a3-bc8f-e247ea1e4c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:36.926909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0a8f368a-04a7-41a3-bc8f-e247ea1e4c38 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:36.928103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0a8f368a-04a7-41a3-bc8f-e247ea1e4c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:37.017058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 73/145] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:37.172811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8f175-af89-4371-ab4f-079671aa2747 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:37.269156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:37.280614 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8f175-af89-4371-ab4f-079671aa2747 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:37.317007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b521c9-046a-4af6-9cb0-cb81bfcf6f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:37.317645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b521c9-046a-4af6-9cb0-cb81bfcf6f1e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:37.317922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b521c9-046a-4af6-9cb0-cb81bfcf6f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:37.318226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b521c9-046a-4af6-9cb0-cb81bfcf6f1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:37.331146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b521c9-046a-4af6-9cb0-cb81bfcf6f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-61b521c9-046a-4af6-9cb0-cb81bfcf6f1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:37.336029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:37.336946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:37.337725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 73/146] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:37.345193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f368a-04a7-41a3-bc8f-e247ea1e4c38 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:37.385386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:37.454621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f368a-04a7-41a3-bc8f-e247ea1e4c38 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:37.494119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25280eb-1094-4744-b305-6c00aace86e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:37.494734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25280eb-1094-4744-b305-6c00aace86e9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:37.495013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25280eb-1094-4744-b305-6c00aace86e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:37.495374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25280eb-1094-4744-b305-6c00aace86e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:37.496043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25280eb-1094-4744-b305-6c00aace86e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:37.496345 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25280eb-1094-4744-b305-6c00aace86e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:37.497807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25280eb-1094-4744-b305-6c00aace86e9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:37.509733 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 74/147] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:37.515762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b521c9-046a-4af6-9cb0-cb81bfcf6f1e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:37.609919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25280eb-1094-4744-b305-6c00aace86e9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:37.669895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b521c9-046a-4af6-9cb0-cb81bfcf6f1e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:37.709587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e43d60-0f4b-42ac-88da-28baea66ab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:37.710213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e43d60-0f4b-42ac-88da-28baea66ab4a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:37.711209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e43d60-0f4b-42ac-88da-28baea66ab4a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:37.711641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e43d60-0f4b-42ac-88da-28baea66ab4a None admin] PATH_INFO: `/v3/users/4a89a68d10724094982d478caa7d87f5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:37.715293 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e43d60-0f4b-42ac-88da-28baea66ab4a None admin] RBAC: Authorizing `identity:get_user(user_id=4a89a68d10724094982d478caa7d87f5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:37.716920 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e43d60-0f4b-42ac-88da-28baea66ab4a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:37.719856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 74/148] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:37 2022] GET /identity/v3/users/4a89a68d10724094982d478caa7d87f5 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:37.725663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e43d60-0f4b-42ac-88da-28baea66ab4a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:37.774312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:37.844717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e43d60-0f4b-42ac-88da-28baea66ab4a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:37.855638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:37.861816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25280eb-1094-4744-b305-6c00aace86e9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:37.902248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db34060-d839-446f-8daa-38759b985818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:37.903493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db34060-d839-446f-8daa-38759b985818 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:37.903805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db34060-d839-446f-8daa-38759b985818 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:37.904084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db34060-d839-446f-8daa-38759b985818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:37.918594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648a507-02e3-4e25-ba7a-e73ceec4c68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:37.919245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648a507-02e3-4e25-ba7a-e73ceec4c68e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:37.919525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648a507-02e3-4e25-ba7a-e73ceec4c68e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:37.919797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648a507-02e3-4e25-ba7a-e73ceec4c68e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:37.931622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648a507-02e3-4e25-ba7a-e73ceec4c68e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-e648a507-02e3-4e25-ba7a-e73ceec4c68e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:37.937071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:37.937920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:37.938698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 75/149] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:13:37.946921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648a507-02e3-4e25-ba7a-e73ceec4c68e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:38.055291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db34060-d839-446f-8daa-38759b985818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:38.057489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db34060-d839-446f-8daa-38759b985818 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:38.092715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648a507-02e3-4e25-ba7a-e73ceec4c68e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:38.103852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:38.167617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64661281-7c2f-4d27-9353-17e4194b7fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:38.168245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64661281-7c2f-4d27-9353-17e4194b7fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:38.168532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64661281-7c2f-4d27-9353-17e4194b7fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:38.168807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64661281-7c2f-4d27-9353-17e4194b7fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:38.169478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64661281-7c2f-4d27-9353-17e4194b7fc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:38.169780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64661281-7c2f-4d27-9353-17e4194b7fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:38.171238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64661281-7c2f-4d27-9353-17e4194b7fc4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:38.185550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 76/150] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:38.192170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64661281-7c2f-4d27-9353-17e4194b7fc4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:38.231508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:38.238339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 75/151] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:38.244215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db34060-d839-446f-8daa-38759b985818 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:38.414588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db34060-d839-446f-8daa-38759b985818 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:38.423919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64661281-7c2f-4d27-9353-17e4194b7fc4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:38.454343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:38.466551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc87c8-1e81-4aa2-a25d-46b0e29f8385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:38.467195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc87c8-1e81-4aa2-a25d-46b0e29f8385 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:38.467515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc87c8-1e81-4aa2-a25d-46b0e29f8385 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:38.467787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc87c8-1e81-4aa2-a25d-46b0e29f8385 None admin] PATH_INFO: `/v3/users/1a2bf170dbcc4f11a4d52fddbe668ebd` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:38.469601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc87c8-1e81-4aa2-a25d-46b0e29f8385 None admin] RBAC: Authorizing `identity:get_user(user_id=1a2bf170dbcc4f11a4d52fddbe668ebd)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:38.471104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc87c8-1e81-4aa2-a25d-46b0e29f8385 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:38.472706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 77/152] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:38 2022] GET /identity/v3/users/1a2bf170dbcc4f11a4d52fddbe668ebd => generated 289 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:38.477372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f034a-aac2-4886-ab65-f220ed181158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:38.478035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f034a-aac2-4886-ab65-f220ed181158 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:38.478320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f034a-aac2-4886-ab65-f220ed181158 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:38.478599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f034a-aac2-4886-ab65-f220ed181158 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:38.479912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc87c8-1e81-4aa2-a25d-46b0e29f8385 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:38.480846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f034a-aac2-4886-ab65-f220ed181158 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:38.482343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f034a-aac2-4886-ab65-f220ed181158 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:38.484820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 76/153] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:38 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:38.501208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f034a-aac2-4886-ab65-f220ed181158 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:38.633722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:38.658407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc87c8-1e81-4aa2-a25d-46b0e29f8385 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:38.677053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f034a-aac2-4886-ab65-f220ed181158 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:38.723539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199efc3c-0352-40b9-bc56-420de2aa8671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:38.724542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199efc3c-0352-40b9-bc56-420de2aa8671 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:38.725037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199efc3c-0352-40b9-bc56-420de2aa8671 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:38.725504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199efc3c-0352-40b9-bc56-420de2aa8671 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:38.727712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d06517-402e-4183-a166-c631bd308b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:38.728530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199efc3c-0352-40b9-bc56-420de2aa8671 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:38.728746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d06517-402e-4183-a166-c631bd308b45 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:38.729216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d06517-402e-4183-a166-c631bd308b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:38.729634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d06517-402e-4183-a166-c631bd308b45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:38.730408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199efc3c-0352-40b9-bc56-420de2aa8671 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:38.730680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d06517-402e-4183-a166-c631bd308b45 None admin] RBAC: Adding query filter params (role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=ad635390dd3d4dcf897fa77044fd970e, user.id=4a89a68d10724094982d478caa7d87f5) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:38.731142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d06517-402e-4183-a166-c631bd308b45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:38.732304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 78/154] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:38 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:38.733557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d06517-402e-4183-a166-c631bd308b45 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:38.738707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199efc3c-0352-40b9-bc56-420de2aa8671 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:38.740761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 77/155] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:38 2022] GET /identity/v3/role_assignments?user.id=4a89a68d10724094982d478caa7d87f5&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=d36487377a664531a7f2e06f2f648a6f => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:38.872836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199efc3c-0352-40b9-bc56-420de2aa8671 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:38.892660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bceb15-1b13-4123-beff-51645a6c8878 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:38.893259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bceb15-1b13-4123-beff-51645a6c8878 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:38.893710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bceb15-1b13-4123-beff-51645a6c8878 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:38.895265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 78/156] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:38.911808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aecb26-08c7-4eda-9ea1-4cfee8413777 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:38.912831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aecb26-08c7-4eda-9ea1-4cfee8413777 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:38.913302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aecb26-08c7-4eda-9ea1-4cfee8413777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:38.943495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e4aecb26-08c7-4eda-9ea1-4cfee8413777 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:38.944388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee2c75-3485-429e-b4c8-e0fcc13a0f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:38.945250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee2c75-3485-429e-b4c8-e0fcc13a0f11 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:38.945713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee2c75-3485-429e-b4c8-e0fcc13a0f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:38.946226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee2c75-3485-429e-b4c8-e0fcc13a0f11 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e/users/1a2bf170dbcc4f11a4d52fddbe668ebd/roles/439053f3a99d43f5abe2fb7dec39df9d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:38.947657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e4aecb26-08c7-4eda-9ea1-4cfee8413777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:38.951892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee2c75-3485-429e-b4c8-e0fcc13a0f11 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad635390dd3d4dcf897fa77044fd970e, user_id=1a2bf170dbcc4f11a4d52fddbe668ebd, role_id=439053f3a99d43f5abe2fb7dec39df9d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:38.953606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee2c75-3485-429e-b4c8-e0fcc13a0f11 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:38.980408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 79/157] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:13:38 2022] PUT /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e/users/1a2bf170dbcc4f11a4d52fddbe668ebd/roles/439053f3a99d43f5abe2fb7dec39df9d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:13:39.024347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:39.048919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 79/158] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:13:39.631814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee2c75-3485-429e-b4c8-e0fcc13a0f11 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:39.655156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:39.755821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee2c75-3485-429e-b4c8-e0fcc13a0f11 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:39.792458 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692cf16-91cb-4e80-8382-cbf33c820fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:39.793524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692cf16-91cb-4e80-8382-cbf33c820fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:39.794141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692cf16-91cb-4e80-8382-cbf33c820fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:39.794626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692cf16-91cb-4e80-8382-cbf33c820fe5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:39.806541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692cf16-91cb-4e80-8382-cbf33c820fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-8692cf16-91cb-4e80-8382-cbf33c820fe5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:39.812416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:39.813319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:39.814181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:39.816113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 80/159] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:13:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:39.822217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aecb26-08c7-4eda-9ea1-4cfee8413777 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:39.901945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f4bd5-76d6-42da-b2b9-d173e9fdf2d5 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:39.902435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f4bd5-76d6-42da-b2b9-d173e9fdf2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:39.902879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f4bd5-76d6-42da-b2b9-d173e9fdf2d5 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:39.904363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 81/160] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:39.909613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f99f28-4695-4210-937e-e567153f362b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:39.910097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f99f28-4695-4210-937e-e567153f362b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:39.910509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f99f28-4695-4210-937e-e567153f362b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:39.946148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-57f99f28-4695-4210-937e-e567153f362b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:39.947766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-57f99f28-4695-4210-937e-e567153f362b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:39.981051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aecb26-08c7-4eda-9ea1-4cfee8413777 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:40.033404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36a8d2-f5a1-4395-80d7-ce07beea5618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:40.034043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36a8d2-f5a1-4395-80d7-ce07beea5618 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:40.034322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36a8d2-f5a1-4395-80d7-ce07beea5618 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:40.034591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36a8d2-f5a1-4395-80d7-ce07beea5618 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:40.035249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36a8d2-f5a1-4395-80d7-ce07beea5618 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:40.035551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36a8d2-f5a1-4395-80d7-ce07beea5618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:40.036928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36a8d2-f5a1-4395-80d7-ce07beea5618 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:40.052421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 80/161] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:13:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:40.065957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36a8d2-f5a1-4395-80d7-ce07beea5618 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:40.068419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 82/162] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:13:40.104091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c3abc-d940-4047-a98a-c120b9d0b4c4 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:40.105133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c3abc-d940-4047-a98a-c120b9d0b4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:40.105792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c3abc-d940-4047-a98a-c120b9d0b4c4 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:40.109017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 83/163] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:40.121590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3856dff-a314-4f50-9adf-cf391b134a60 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:40.122194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3856dff-a314-4f50-9adf-cf391b134a60 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:40.122666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3856dff-a314-4f50-9adf-cf391b134a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:40.142603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:40.147703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:40.152163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b3856dff-a314-4f50-9adf-cf391b134a60 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:40.154736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b3856dff-a314-4f50-9adf-cf391b134a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:40.256206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36a8d2-f5a1-4395-80d7-ce07beea5618 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:40.287019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 84/164] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 21 05:13:40.317447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2836e-b490-47ea-a673-3786349a7a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:40.318626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2836e-b490-47ea-a673-3786349a7a85 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:40.319022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2836e-b490-47ea-a673-3786349a7a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:40.319309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2836e-b490-47ea-a673-3786349a7a85 None admin] PATH_INFO: `/v3/users/4a89a68d10724094982d478caa7d87f5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:40.320514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2836e-b490-47ea-a673-3786349a7a85 None admin] RBAC: Authorizing `identity:get_user(user_id=4a89a68d10724094982d478caa7d87f5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:40.321977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2836e-b490-47ea-a673-3786349a7a85 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:40.324016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 81/165] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:40 2022] GET /identity/v3/users/4a89a68d10724094982d478caa7d87f5 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:40.329621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3856dff-a314-4f50-9adf-cf391b134a60 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:40.447660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3856dff-a314-4f50-9adf-cf391b134a60 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:40.492007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eaff9-f81d-4fbd-8198-9630e1f964c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:40.492952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eaff9-f81d-4fbd-8198-9630e1f964c1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:40.493442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eaff9-f81d-4fbd-8198-9630e1f964c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:40.493918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eaff9-f81d-4fbd-8198-9630e1f964c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:40.501750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:40.578738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3eaff9-f81d-4fbd-8198-9630e1f964c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:40.580213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3eaff9-f81d-4fbd-8198-9630e1f964c1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:40.694154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 85/166] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:40.700069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2836e-b490-47ea-a673-3786349a7a85 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:40.771128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eaff9-f81d-4fbd-8198-9630e1f964c1 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:40.848230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:40.857322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2836e-b490-47ea-a673-3786349a7a85 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:40.874473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:40.899529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5dd6ef-b4ba-4554-8e8b-211885aa9b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:40.900169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5dd6ef-b4ba-4554-8e8b-211885aa9b45 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:40.900450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5dd6ef-b4ba-4554-8e8b-211885aa9b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:40.900729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5dd6ef-b4ba-4554-8e8b-211885aa9b45 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:40.902439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5dd6ef-b4ba-4554-8e8b-211885aa9b45 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:40.903945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5dd6ef-b4ba-4554-8e8b-211885aa9b45 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:40.905599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 82/167] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:40 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:40.906474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5dd6ef-b4ba-4554-8e8b-211885aa9b45 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:41.003992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eaff9-f81d-4fbd-8198-9630e1f964c1 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:41.050335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5dd6ef-b4ba-4554-8e8b-211885aa9b45 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:41.072489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28c282-f0de-4bc9-8450-91c9e65c2d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:41.073123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28c282-f0de-4bc9-8450-91c9e65c2d61 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.073397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28c282-f0de-4bc9-8450-91c9e65c2d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.073663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28c282-f0de-4bc9-8450-91c9e65c2d61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.096208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28c282-f0de-4bc9-8450-91c9e65c2d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-4a28c282-f0de-4bc9-8450-91c9e65c2d61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:41.101023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:41.102049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:41.102795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 86/168] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:41.103480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28c282-f0de-4bc9-8450-91c9e65c2d61 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:41.118347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32348227-f21a-4f8c-ab6f-57214b0983b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:41.118994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32348227-f21a-4f8c-ab6f-57214b0983b1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.119277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32348227-f21a-4f8c-ab6f-57214b0983b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.119549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32348227-f21a-4f8c-ab6f-57214b0983b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.138382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32348227-f21a-4f8c-ab6f-57214b0983b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-32348227-f21a-4f8c-ab6f-57214b0983b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:41.143819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:41.144685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:41.145458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 83/169] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:41.146843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32348227-f21a-4f8c-ab6f-57214b0983b1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:41.221161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:41.291628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:41.298185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28c282-f0de-4bc9-8450-91c9e65c2d61 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:41.315003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32348227-f21a-4f8c-ab6f-57214b0983b1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:41.362342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6917d4-03c8-4ba1-b651-69d152068f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:41.363210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6917d4-03c8-4ba1-b651-69d152068f53 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.363485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6917d4-03c8-4ba1-b651-69d152068f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.363755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6917d4-03c8-4ba1-b651-69d152068f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.364473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6917d4-03c8-4ba1-b651-69d152068f53 None admin] RBAC: Adding query filter params (role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=ad635390dd3d4dcf897fa77044fd970e, user.id=4a89a68d10724094982d478caa7d87f5) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:41.364777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6917d4-03c8-4ba1-b651-69d152068f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:41.366231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6917d4-03c8-4ba1-b651-69d152068f53 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:41.371972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e4334-53fe-4d04-934b-f7f2d7979509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:41.372581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e4334-53fe-4d04-934b-f7f2d7979509 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.372855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e4334-53fe-4d04-934b-f7f2d7979509 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.373124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e4334-53fe-4d04-934b-f7f2d7979509 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.373816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e4334-53fe-4d04-934b-f7f2d7979509 None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:41.374134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e4334-53fe-4d04-934b-f7f2d7979509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:41.375489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e4334-53fe-4d04-934b-f7f2d7979509 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:41.381284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 87/170] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:41 2022] GET /identity/v3/role_assignments?user.id=4a89a68d10724094982d478caa7d87f5&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:41.382417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ee82c-c32c-4f12-8444-5c13eaf08516 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.382865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ee82c-c32c-4f12-8444-5c13eaf08516 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.383135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ee82c-c32c-4f12-8444-5c13eaf08516 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.384453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 88/171] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:41.384999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ee82c-c32c-4f12-8444-5c13eaf08516 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:41.395466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 84/172] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:41.396650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e959483-3a05-4cb7-9cb6-aad3f8b1f814 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.396984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e959483-3a05-4cb7-9cb6-aad3f8b1f814 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.397259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e959483-3a05-4cb7-9cb6-aad3f8b1f814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.442956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4e959483-3a05-4cb7-9cb6-aad3f8b1f814 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:41.444174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4e959483-3a05-4cb7-9cb6-aad3f8b1f814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:41.555434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ee82c-c32c-4f12-8444-5c13eaf08516 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:41.611905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 85/173] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:41.612774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e959483-3a05-4cb7-9cb6-aad3f8b1f814 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:41.635770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af51fe-2e23-4088-a632-5b0e4d3ffac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:41.636390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af51fe-2e23-4088-a632-5b0e4d3ffac0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.636663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af51fe-2e23-4088-a632-5b0e4d3ffac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.636932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af51fe-2e23-4088-a632-5b0e4d3ffac0 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.637594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af51fe-2e23-4088-a632-5b0e4d3ffac0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:41.637893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af51fe-2e23-4088-a632-5b0e4d3ffac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:41.639868 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af51fe-2e23-4088-a632-5b0e4d3ffac0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:41.660667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 89/174] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:41.666774 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af51fe-2e23-4088-a632-5b0e4d3ffac0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:41.687966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:41.725052 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:41.774615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e959483-3a05-4cb7-9cb6-aad3f8b1f814 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:41.850547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af51fe-2e23-4088-a632-5b0e4d3ffac0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:41.867181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4f1fe-4939-416f-8bf3-c19949a72899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:41.867830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4f1fe-4939-416f-8bf3-c19949a72899 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.868107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4f1fe-4939-416f-8bf3-c19949a72899 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.868378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4f1fe-4939-416f-8bf3-c19949a72899 None admin] PATH_INFO: `/v3/users/4a89a68d10724094982d478caa7d87f5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.874633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4f1fe-4939-416f-8bf3-c19949a72899 None admin] RBAC: Authorizing `identity:get_user(user_id=4a89a68d10724094982d478caa7d87f5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:41.876078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4f1fe-4939-416f-8bf3-c19949a72899 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:41.877686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 86/175] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:41 2022] GET /identity/v3/users/4a89a68d10724094982d478caa7d87f5 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:41.878905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cee95-4d57-49d4-b955-f1ed5ef44c9c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.879139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cee95-4d57-49d4-b955-f1ed5ef44c9c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.879411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cee95-4d57-49d4-b955-f1ed5ef44c9c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.880603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 87/176] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:41.884199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cee95-4d57-49d4-b955-f1ed5ef44c9c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:41.892930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e8d85-ea6d-4468-a133-41f9b4452916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:41.893582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e8d85-ea6d-4468-a133-41f9b4452916 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.893861 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e8d85-ea6d-4468-a133-41f9b4452916 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.894171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e8d85-ea6d-4468-a133-41f9b4452916 None admin] PATH_INFO: `/v3/users/493af8396fb3419ea552b2a7642f7a68` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.895489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e8d85-ea6d-4468-a133-41f9b4452916 None admin] RBAC: Authorizing `identity:get_user(user_id=493af8396fb3419ea552b2a7642f7a68)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:41.897146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e8d85-ea6d-4468-a133-41f9b4452916 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:41.899129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 90/177] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:41 2022] GET /identity/v3/users/493af8396fb3419ea552b2a7642f7a68 => generated 247 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:41.900053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8785d09-8864-468c-9dd4-1bbd6e1fcea0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:41.900333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8785d09-8864-468c-9dd4-1bbd6e1fcea0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:41.900670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8785d09-8864-468c-9dd4-1bbd6e1fcea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:41.920089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e8785d09-8864-468c-9dd4-1bbd6e1fcea0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:41.921423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e8785d09-8864-468c-9dd4-1bbd6e1fcea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:41.996271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 91/178] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:41.997157 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8785d09-8864-468c-9dd4-1bbd6e1fcea0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:42.034480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:42.129468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8785d09-8864-468c-9dd4-1bbd6e1fcea0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:42.146959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:42.159878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cee95-4d57-49d4-b955-f1ed5ef44c9c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:42.172044 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57727334-4f5f-4f9e-9f5b-3dd1daa9185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:42.172924 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57727334-4f5f-4f9e-9f5b-3dd1daa9185b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.173378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57727334-4f5f-4f9e-9f5b-3dd1daa9185b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.173823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57727334-4f5f-4f9e-9f5b-3dd1daa9185b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.240589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1f1c2-a3c3-44b1-bcb8-00f0082a602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:42.241197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1f1c2-a3c3-44b1-bcb8-00f0082a602b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.241472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1f1c2-a3c3-44b1-bcb8-00f0082a602b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.241742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1f1c2-a3c3-44b1-bcb8-00f0082a602b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.247175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57727334-4f5f-4f9e-9f5b-3dd1daa9185b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:42.248874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57727334-4f5f-4f9e-9f5b-3dd1daa9185b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:42.270911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:42.321792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1f1c2-a3c3-44b1-bcb8-00f0082a602b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:42.324854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1f1c2-a3c3-44b1-bcb8-00f0082a602b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:42.361381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 92/179] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:42.362075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57727334-4f5f-4f9e-9f5b-3dd1daa9185b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:42.436394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:42.439936 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 88/180] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 05:13:42.441420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584f2ed-9107-4117-8b82-f24ebf47590d None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.441864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584f2ed-9107-4117-8b82-f24ebf47590d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.442331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584f2ed-9107-4117-8b82-f24ebf47590d None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.443699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 89/181] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:42.444234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584f2ed-9107-4117-8b82-f24ebf47590d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:42.471270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57727334-4f5f-4f9e-9f5b-3dd1daa9185b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:42.491528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:42.506416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669182f-ff30-461d-bb19-bee9a192c0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:42.507045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669182f-ff30-461d-bb19-bee9a192c0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.507317 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669182f-ff30-461d-bb19-bee9a192c0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.507588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669182f-ff30-461d-bb19-bee9a192c0e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.517712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669182f-ff30-461d-bb19-bee9a192c0e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5669182f-ff30-461d-bb19-bee9a192c0e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:42.522506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:42.523371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:42.524151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 93/182] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:13:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:42.529003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669182f-ff30-461d-bb19-bee9a192c0e3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:42.544484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584f2ed-9107-4117-8b82-f24ebf47590d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:42.578155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf368e2-efd0-4d51-aa7d-922a141688ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:42.578771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf368e2-efd0-4d51-aa7d-922a141688ba None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.579041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf368e2-efd0-4d51-aa7d-922a141688ba None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.579311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf368e2-efd0-4d51-aa7d-922a141688ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.580387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf368e2-efd0-4d51-aa7d-922a141688ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:42.581818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf368e2-efd0-4d51-aa7d-922a141688ba None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:42.585993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 90/183] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:42.585993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af44e-e0aa-4a5d-8e86-97fbb1bfcdab None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.585993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af44e-e0aa-4a5d-8e86-97fbb1bfcdab None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.585993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af44e-e0aa-4a5d-8e86-97fbb1bfcdab None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.586327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 91/184] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:42.586875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af44e-e0aa-4a5d-8e86-97fbb1bfcdab None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:42.633211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669182f-ff30-461d-bb19-bee9a192c0e3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:42.654115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:42.666683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce106d01-989a-4ff0-b818-c5f9aee050cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:42.667301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce106d01-989a-4ff0-b818-c5f9aee050cd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.667574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce106d01-989a-4ff0-b818-c5f9aee050cd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.667840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce106d01-989a-4ff0-b818-c5f9aee050cd None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.669241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce106d01-989a-4ff0-b818-c5f9aee050cd None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:42.670443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce106d01-989a-4ff0-b818-c5f9aee050cd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:42.672045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 94/185] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:42 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:42.673159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4338d85-1086-42e1-86a9-77d3b8e452d1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.673401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4338d85-1086-42e1-86a9-77d3b8e452d1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.673669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4338d85-1086-42e1-86a9-77d3b8e452d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.690017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af44e-e0aa-4a5d-8e86-97fbb1bfcdab None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:42.691400 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b4338d85-1086-42e1-86a9-77d3b8e452d1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:42.692566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b4338d85-1086-42e1-86a9-77d3b8e452d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:42.723177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b9092-bb20-475a-81b5-04d727d049fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:42.723805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b9092-bb20-475a-81b5-04d727d049fd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.724084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b9092-bb20-475a-81b5-04d727d049fd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.724354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b9092-bb20-475a-81b5-04d727d049fd None admin] PATH_INFO: `/v3/projects/169129b4cc1b4357ae8cdd36a8665553` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.725452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b9092-bb20-475a-81b5-04d727d049fd None admin] RBAC: Authorizing `identity:get_project(project_id=169129b4cc1b4357ae8cdd36a8665553)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:42.726925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b9092-bb20-475a-81b5-04d727d049fd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:42.728537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 92/186] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:42 2022] GET /identity/v3/projects/169129b4cc1b4357ae8cdd36a8665553 => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:42.729291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b9092-bb20-475a-81b5-04d727d049fd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:42.743145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:42.767109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 95/187] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:42.767911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4338d85-1086-42e1-86a9-77d3b8e452d1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:42.834722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b9092-bb20-475a-81b5-04d727d049fd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:42.868751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4338d85-1086-42e1-86a9-77d3b8e452d1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:42.870442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59268434-d133-41fa-ad50-5956b8f4b7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:42.871046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59268434-d133-41fa-ad50-5956b8f4b7cc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.871322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59268434-d133-41fa-ad50-5956b8f4b7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.871589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59268434-d133-41fa-ad50-5956b8f4b7cc None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.872231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59268434-d133-41fa-ad50-5956b8f4b7cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:42.872528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59268434-d133-41fa-ad50-5956b8f4b7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:42.874510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59268434-d133-41fa-ad50-5956b8f4b7cc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:42.876964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:42.888119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 93/188] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:42.889279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccc8ca-c05d-40be-af13-baf760382a65 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.889564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccc8ca-c05d-40be-af13-baf760382a65 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.889840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccc8ca-c05d-40be-af13-baf760382a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.906542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36eb64e-64fc-434a-8f9f-0042060efaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:42.906876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-85ccc8ca-c05d-40be-af13-baf760382a65 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:42.907149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36eb64e-64fc-434a-8f9f-0042060efaf5 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:42.907427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36eb64e-64fc-434a-8f9f-0042060efaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:42.907697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36eb64e-64fc-434a-8f9f-0042060efaf5 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:42.908036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-85ccc8ca-c05d-40be-af13-baf760382a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:42.908337 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36eb64e-64fc-434a-8f9f-0042060efaf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:42.909767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36eb64e-64fc-434a-8f9f-0042060efaf5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:42.932295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 96/189] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:13:42 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:42.933106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36eb64e-64fc-434a-8f9f-0042060efaf5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:42.992764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:42.996290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 94/190] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:42.997542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccc8ca-c05d-40be-af13-baf760382a65 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:43.128396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccc8ca-c05d-40be-af13-baf760382a65 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:43.132579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36eb64e-64fc-434a-8f9f-0042060efaf5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:43.159063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:43.169688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148e94a-70b7-485b-b3e8-263d1bffe589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:43.170987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148e94a-70b7-485b-b3e8-263d1bffe589 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:43.171269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148e94a-70b7-485b-b3e8-263d1bffe589 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:43.171540 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148e94a-70b7-485b-b3e8-263d1bffe589 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:43.172267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148e94a-70b7-485b-b3e8-263d1bffe589 None admin] RBAC: Adding query filter params (role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=169129b4cc1b4357ae8cdd36a8665553, user.id=4a89a68d10724094982d478caa7d87f5) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:43.172578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148e94a-70b7-485b-b3e8-263d1bffe589 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:43.174049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148e94a-70b7-485b-b3e8-263d1bffe589 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:43.176384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b054b71-992c-43bd-a504-96d7aa786f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:43.176991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b054b71-992c-43bd-a504-96d7aa786f8b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:43.177341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b054b71-992c-43bd-a504-96d7aa786f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:43.177623 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b054b71-992c-43bd-a504-96d7aa786f8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:43.178408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b054b71-992c-43bd-a504-96d7aa786f8b None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=ad635390dd3d4dcf897fa77044fd970e, user.id=493af8396fb3419ea552b2a7642f7a68) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:43.178717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b054b71-992c-43bd-a504-96d7aa786f8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:43.180205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b054b71-992c-43bd-a504-96d7aa786f8b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:43.181452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 95/191] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:42 2022] GET /identity/v3/role_assignments?user.id=4a89a68d10724094982d478caa7d87f5&scope.project.id=169129b4cc1b4357ae8cdd36a8665553&role.id=d36487377a664531a7f2e06f2f648a6f => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:43.182530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148e94a-70b7-485b-b3e8-263d1bffe589 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:43.193056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 97/192] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:42 2022] GET /identity/v3/role_assignments?user.id=493af8396fb3419ea552b2a7642f7a68&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:43.194812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b054b71-992c-43bd-a504-96d7aa786f8b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:43.311460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148e94a-70b7-485b-b3e8-263d1bffe589 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:43.329328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:43.334622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b054b71-992c-43bd-a504-96d7aa786f8b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:43.361692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4c84d-5835-4243-89d9-36d12c6f4906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:43.361692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4c84d-5835-4243-89d9-36d12c6f4906 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:43.361692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4c84d-5835-4243-89d9-36d12c6f4906 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:43.361692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4c84d-5835-4243-89d9-36d12c6f4906 None admin] PATH_INFO: `/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:43.386426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4c84d-5835-4243-89d9-36d12c6f4906 None admin] RBAC: Authorizing `identity:get_user(user_id=a36e7ae1cb5d4a3f9c474d465c436e6d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:43.388571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4c84d-5835-4243-89d9-36d12c6f4906 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:43.389532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3be78-600c-4e1a-93b2-5184ca12f434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:43.390209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3be78-600c-4e1a-93b2-5184ca12f434 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:43.390485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3be78-600c-4e1a-93b2-5184ca12f434 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:43.390765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3be78-600c-4e1a-93b2-5184ca12f434 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:43.391570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 96/193] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:43 2022] GET /identity/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:43.393492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4c84d-5835-4243-89d9-36d12c6f4906 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:43.409477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3be78-600c-4e1a-93b2-5184ca12f434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-f7b3be78-600c-4e1a-93b2-5184ca12f434 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:43.417982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:43.418932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 98/194] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:13:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:43.420978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3be78-600c-4e1a-93b2-5184ca12f434 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:43.480948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:43.553184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4c84d-5835-4243-89d9-36d12c6f4906 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:43.569078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3be78-600c-4e1a-93b2-5184ca12f434 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:43.614501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d4c8-7378-4f1f-a381-865a064ed65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:43.614918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d4c8-7378-4f1f-a381-865a064ed65a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:43.615200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d4c8-7378-4f1f-a381-865a064ed65a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:43.615476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d4c8-7378-4f1f-a381-865a064ed65a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:43.628115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d3811-2c0d-4136-acbb-462dd9babd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:43.628731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d3811-2c0d-4136-acbb-462dd9babd14 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:43.629009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d3811-2c0d-4136-acbb-462dd9babd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:43.629282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d3811-2c0d-4136-acbb-462dd9babd14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:43.632457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383d4c8-7378-4f1f-a381-865a064ed65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1383d4c8-7378-4f1f-a381-865a064ed65a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:43.637670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:43.640254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:43.641055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:43.643425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 97/195] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:43.644543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d4c8-7378-4f1f-a381-865a064ed65a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:43.659658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:43.702770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d3811-2c0d-4136-acbb-462dd9babd14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:43.704244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d3811-2c0d-4136-acbb-462dd9babd14 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:43.763491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d4c8-7378-4f1f-a381-865a064ed65a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:43.777507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:43.808021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dbbd5-2ccd-410e-8693-07109f7f615e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:43.809470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dbbd5-2ccd-410e-8693-07109f7f615e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:43.810007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dbbd5-2ccd-410e-8693-07109f7f615e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:43.810421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dbbd5-2ccd-410e-8693-07109f7f615e None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:43.811346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819dbbd5-2ccd-410e-8693-07109f7f615e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:43.811787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819dbbd5-2ccd-410e-8693-07109f7f615e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:43.813384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819dbbd5-2ccd-410e-8693-07109f7f615e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:43.823122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 98/196] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:13:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:43.824374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dbbd5-2ccd-410e-8693-07109f7f615e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:43.834269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 99/197] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:43.835107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d3811-2c0d-4136-acbb-462dd9babd14 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:43.912946 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:43.948629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dbbd5-2ccd-410e-8693-07109f7f615e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:43.964293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d3811-2c0d-4136-acbb-462dd9babd14 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:43.988653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66deb406-17ef-404f-93a9-aca73032ff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:43.989269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66deb406-17ef-404f-93a9-aca73032ff5b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:43.989550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66deb406-17ef-404f-93a9-aca73032ff5b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:43.989832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66deb406-17ef-404f-93a9-aca73032ff5b None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:43.990658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66deb406-17ef-404f-93a9-aca73032ff5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:43.990960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66deb406-17ef-404f-93a9-aca73032ff5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:43.992317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66deb406-17ef-404f-93a9-aca73032ff5b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:44.002784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 99/198] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:44.004628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a5e4d-cf8d-48d5-ab9c-022c8c4b85fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:44.005548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a5e4d-cf8d-48d5-ab9c-022c8c4b85fe None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:44.006104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a5e4d-cf8d-48d5-ab9c-022c8c4b85fe None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:44.006575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a5e4d-cf8d-48d5-ab9c-022c8c4b85fe None admin] PATH_INFO: `/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:44.006809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66deb406-17ef-404f-93a9-aca73032ff5b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:44.008133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a5e4d-cf8d-48d5-ab9c-022c8c4b85fe None admin] RBAC: Authorizing `identity:get_user(user_id=a36e7ae1cb5d4a3f9c474d465c436e6d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:44.009985 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a5e4d-cf8d-48d5-ab9c-022c8c4b85fe None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:44.011822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 100/199] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:43 2022] GET /identity/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:44.012887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a5e4d-cf8d-48d5-ab9c-022c8c4b85fe None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:44.051868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:44.121559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a5e4d-cf8d-48d5-ab9c-022c8c4b85fe None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:44.125626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66deb406-17ef-404f-93a9-aca73032ff5b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:44.160229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b2f0e-1b35-46ff-9462-4406eaddda07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:44.161519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b2f0e-1b35-46ff-9462-4406eaddda07 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:44.162129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b2f0e-1b35-46ff-9462-4406eaddda07 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:44.162622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b2f0e-1b35-46ff-9462-4406eaddda07 None admin] PATH_INFO: `/v3/users/1a2bf170dbcc4f11a4d52fddbe668ebd` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:44.163632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:44.163738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b5e8d-815b-4f56-b8d1-09e7f38c2e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:44.164329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b5e8d-815b-4f56-b8d1-09e7f38c2e8a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:44.164608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b5e8d-815b-4f56-b8d1-09e7f38c2e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:44.164875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b5e8d-815b-4f56-b8d1-09e7f38c2e8a None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:44.166032 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b5e8d-815b-4f56-b8d1-09e7f38c2e8a None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:44.167088 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b2f0e-1b35-46ff-9462-4406eaddda07 None admin] RBAC: Authorizing `identity:get_user(user_id=1a2bf170dbcc4f11a4d52fddbe668ebd)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:44.167457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b5e8d-815b-4f56-b8d1-09e7f38c2e8a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:44.168926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b2f0e-1b35-46ff-9462-4406eaddda07 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:44.169400 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 100/200] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:44 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:44.170685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b5e8d-815b-4f56-b8d1-09e7f38c2e8a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:44.172372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 101/201] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:44 2022] GET /identity/v3/users/1a2bf170dbcc4f11a4d52fddbe668ebd => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:44.182059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b2f0e-1b35-46ff-9462-4406eaddda07 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:44.291184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b5e8d-815b-4f56-b8d1-09e7f38c2e8a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:44.292889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:44.314922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b2f0e-1b35-46ff-9462-4406eaddda07 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:44.335561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc045b1-0b65-450b-b1b3-26e9c5688147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:44.336282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc045b1-0b65-450b-b1b3-26e9c5688147 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:44.336625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc045b1-0b65-450b-b1b3-26e9c5688147 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:44.336960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc045b1-0b65-450b-b1b3-26e9c5688147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:44.355228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2be200-c31b-484c-994c-945ffef3a8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:44.356174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2be200-c31b-484c-994c-945ffef3a8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:44.356587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2be200-c31b-484c-994c-945ffef3a8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:44.357012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2be200-c31b-484c-994c-945ffef3a8d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:44.358434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2be200-c31b-484c-994c-945ffef3a8d4 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user.id=a36e7ae1cb5d4a3f9c474d465c436e6d) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:44.358933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2be200-c31b-484c-994c-945ffef3a8d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:44.360653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2be200-c31b-484c-994c-945ffef3a8d4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:44.374594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 102/202] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:44 2022] GET /identity/v3/role_assignments?user.id=a36e7ae1cb5d4a3f9c474d465c436e6d&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:44.375821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2be200-c31b-484c-994c-945ffef3a8d4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:44.415084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc045b1-0b65-450b-b1b3-26e9c5688147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:44.418313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc045b1-0b65-450b-b1b3-26e9c5688147 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:44.493355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:44.529337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 101/203] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:44.537476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc045b1-0b65-450b-b1b3-26e9c5688147 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:44.539607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:44.568894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2be200-c31b-484c-994c-945ffef3a8d4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:44.639413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125a933-8543-4e91-96ef-a1305ba4991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:44.639954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125a933-8543-4e91-96ef-a1305ba4991c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:44.640230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125a933-8543-4e91-96ef-a1305ba4991c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:44.640501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125a933-8543-4e91-96ef-a1305ba4991c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:44.722022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc045b1-0b65-450b-b1b3-26e9c5688147 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:44.734096 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125a933-8543-4e91-96ef-a1305ba4991c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:44.735603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125a933-8543-4e91-96ef-a1305ba4991c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:44.790147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ee9e4-0f65-457a-a94c-0de0e0e8e5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:44.790798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ee9e4-0f65-457a-a94c-0de0e0e8e5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:44.791078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ee9e4-0f65-457a-a94c-0de0e0e8e5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:44.791355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ee9e4-0f65-457a-a94c-0de0e0e8e5a8 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:44.792615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ee9e4-0f65-457a-a94c-0de0e0e8e5a8 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:44.794089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ee9e4-0f65-457a-a94c-0de0e0e8e5a8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:44.796506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 102/204] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:44 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:44.802477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ee9e4-0f65-457a-a94c-0de0e0e8e5a8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:44.848090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:44.944165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:44.955349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 103/205] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:44.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125a933-8543-4e91-96ef-a1305ba4991c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:44.966266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ee9e4-0f65-457a-a94c-0de0e0e8e5a8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:45.004498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b38229-460e-4bc9-9630-77b37cfedb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:45.005109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b38229-460e-4bc9-9630-77b37cfedb95 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:45.005390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b38229-460e-4bc9-9630-77b37cfedb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:45.005662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b38229-460e-4bc9-9630-77b37cfedb95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:45.006417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b38229-460e-4bc9-9630-77b37cfedb95 None admin] RBAC: Adding query filter params (role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user.id=a36e7ae1cb5d4a3f9c474d465c436e6d) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:45.007126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b38229-460e-4bc9-9630-77b37cfedb95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:45.008558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b38229-460e-4bc9-9630-77b37cfedb95 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:45.014833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 103/206] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:44 2022] GET /identity/v3/role_assignments?user.id=a36e7ae1cb5d4a3f9c474d465c436e6d&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:45.157266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125a933-8543-4e91-96ef-a1305ba4991c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:45.205079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05c097-cd65-4840-9621-a70c7d1bfdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:45.205651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05c097-cd65-4840-9621-a70c7d1bfdac None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:45.205950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05c097-cd65-4840-9621-a70c7d1bfdac None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:45.206344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05c097-cd65-4840-9621-a70c7d1bfdac None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:45.207515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05c097-cd65-4840-9621-a70c7d1bfdac None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:45.208998 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05c097-cd65-4840-9621-a70c7d1bfdac None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:45.215378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 104/207] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:44 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:45.221409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b38229-460e-4bc9-9630-77b37cfedb95 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:45.336317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b38229-460e-4bc9-9630-77b37cfedb95 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:45.372007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce65be-57a0-4b47-bfe4-1b59aade4911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:45.372617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce65be-57a0-4b47-bfe4-1b59aade4911 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:45.372896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce65be-57a0-4b47-bfe4-1b59aade4911 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:45.373169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce65be-57a0-4b47-bfe4-1b59aade4911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:45.373925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce65be-57a0-4b47-bfe4-1b59aade4911 None admin] RBAC: Adding query filter params (role.id=439053f3a99d43f5abe2fb7dec39df9d, scope.project.id=ad635390dd3d4dcf897fa77044fd970e, user.id=1a2bf170dbcc4f11a4d52fddbe668ebd) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:45.374263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce65be-57a0-4b47-bfe4-1b59aade4911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:45.375762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce65be-57a0-4b47-bfe4-1b59aade4911 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:45.377650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:45.381897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 104/208] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:45 2022] GET /identity/v3/role_assignments?user.id=1a2bf170dbcc4f11a4d52fddbe668ebd&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=439053f3a99d43f5abe2fb7dec39df9d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:45.835945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b446723-c89b-4f34-a418-a6738a8df6af None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:45.836333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b446723-c89b-4f34-a418-a6738a8df6af None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:45.836505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b446723-c89b-4f34-a418-a6738a8df6af None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:45.837852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 105/209] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:45.843441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1edb9-bbb0-4e39-868f-543bc2a3029f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:45.843677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1edb9-bbb0-4e39-868f-543bc2a3029f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:45.843957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1edb9-bbb0-4e39-868f-543bc2a3029f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:45.864877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c10bb-a54a-4078-98ca-d33e30b5bb43 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:45.865369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c10bb-a54a-4078-98ca-d33e30b5bb43 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:45.865441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c10bb-a54a-4078-98ca-d33e30b5bb43 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:45.865677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-56f1edb9-bbb0-4e39-868f-543bc2a3029f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:45.866909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-56f1edb9-bbb0-4e39-868f-543bc2a3029f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:45.867040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 106/210] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:45.872251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7156b-288d-42f3-8f5a-dfaaedb326a7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:45.872525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7156b-288d-42f3-8f5a-dfaaedb326a7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:45.872792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7156b-288d-42f3-8f5a-dfaaedb326a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:45.898660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-86d7156b-288d-42f3-8f5a-dfaaedb326a7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:45.900819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-86d7156b-288d-42f3-8f5a-dfaaedb326a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:45.924619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:45.949041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 105/211] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:45.996204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 107/212] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:46.245802 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1edb9-bbb0-4e39-868f-543bc2a3029f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:46.275094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7156b-288d-42f3-8f5a-dfaaedb326a7 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:46.367647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1edb9-bbb0-4e39-868f-543bc2a3029f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:46.401967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7156b-288d-42f3-8f5a-dfaaedb326a7 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:46.404365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:46.415200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394eb405-29e9-42df-8374-6166e0e880b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:46.416640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394eb405-29e9-42df-8374-6166e0e880b3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:46.417154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394eb405-29e9-42df-8374-6166e0e880b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:46.418049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394eb405-29e9-42df-8374-6166e0e880b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:46.419625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394eb405-29e9-42df-8374-6166e0e880b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:46.422877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394eb405-29e9-42df-8374-6166e0e880b3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:46.425604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 106/213] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:46.430880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394eb405-29e9-42df-8374-6166e0e880b3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:46.444023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7083e-f25b-4e6f-b64c-3ce8ef819816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:46.445137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7083e-f25b-4e6f-b64c-3ce8ef819816 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:46.445445 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7083e-f25b-4e6f-b64c-3ce8ef819816 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:46.445724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7083e-f25b-4e6f-b64c-3ce8ef819816 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:46.460823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7083e-f25b-4e6f-b64c-3ce8ef819816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-eed7083e-f25b-4e6f-b64c-3ce8ef819816 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:46.465912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:46.466769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:46.468140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 108/214] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:13:46.474985 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7083e-f25b-4e6f-b64c-3ce8ef819816 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:46.496317 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:46.559168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394eb405-29e9-42df-8374-6166e0e880b3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:46.589095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7083e-f25b-4e6f-b64c-3ce8ef819816 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:46.597980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe60da-47b2-48a5-80c4-338c78099b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:46.598800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe60da-47b2-48a5-80c4-338c78099b72 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:46.599232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe60da-47b2-48a5-80c4-338c78099b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:46.599656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe60da-47b2-48a5-80c4-338c78099b72 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:46.600455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe60da-47b2-48a5-80c4-338c78099b72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:46.602082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe60da-47b2-48a5-80c4-338c78099b72 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:46.627333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7042b-19cf-4cd8-a877-3c93473196a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:46.627957 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7042b-19cf-4cd8-a877-3c93473196a4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:46.628215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7042b-19cf-4cd8-a877-3c93473196a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:46.628483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7042b-19cf-4cd8-a877-3c93473196a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:46.629148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7042b-19cf-4cd8-a877-3c93473196a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:46.629441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7042b-19cf-4cd8-a877-3c93473196a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:46.630131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 107/215] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:13:46 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:46.638385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7042b-19cf-4cd8-a877-3c93473196a4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:46.648535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 109/216] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:46.654441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe60da-47b2-48a5-80c4-338c78099b72 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:46.712430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:46.778945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe60da-47b2-48a5-80c4-338c78099b72 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:46.818937 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cbace-247c-47dd-aba1-43e9a0e1fee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:46.819986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cbace-247c-47dd-aba1-43e9a0e1fee2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:46.820512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cbace-247c-47dd-aba1-43e9a0e1fee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:46.820995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cbace-247c-47dd-aba1-43e9a0e1fee2 None admin] PATH_INFO: `/v3/users/dcf2e6a2e4e34d41849ea6df285cd403` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:46.837377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cbace-247c-47dd-aba1-43e9a0e1fee2 None admin] RBAC: Authorizing `identity:get_user(user_id=dcf2e6a2e4e34d41849ea6df285cd403)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:46.839235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cbace-247c-47dd-aba1-43e9a0e1fee2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:46.841438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 108/217] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:46 2022] GET /identity/v3/users/dcf2e6a2e4e34d41849ea6df285cd403 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:46.847851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7042b-19cf-4cd8-a877-3c93473196a4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:46.872985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eed54e-b20a-4b6e-98a8-4e52c2c8281f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:46.873341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eed54e-b20a-4b6e-98a8-4e52c2c8281f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:46.873612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eed54e-b20a-4b6e-98a8-4e52c2c8281f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:46.875505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 109/218] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:46.881219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39136851-3e1a-4700-b90f-127058d94bb3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:46.881842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39136851-3e1a-4700-b90f-127058d94bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:46.882147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39136851-3e1a-4700-b90f-127058d94bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:46.908299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-39136851-3e1a-4700-b90f-127058d94bb3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:46.909923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-39136851-3e1a-4700-b90f-127058d94bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:46.943900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:46.966720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7042b-19cf-4cd8-a877-3c93473196a4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:46.994117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 110/219] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:13:47.011586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5dff2-34a8-46ae-8ac3-7d754d1948e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:47.012201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5dff2-34a8-46ae-8ac3-7d754d1948e0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:47.012476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5dff2-34a8-46ae-8ac3-7d754d1948e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:47.013027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5dff2-34a8-46ae-8ac3-7d754d1948e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:47.089238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5dff2-34a8-46ae-8ac3-7d754d1948e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:47.091074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5dff2-34a8-46ae-8ac3-7d754d1948e0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:47.171842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:47.202300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 110/220] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:47.208302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39136851-3e1a-4700-b90f-127058d94bb3 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:47.259922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:47.352826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39136851-3e1a-4700-b90f-127058d94bb3 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:47.391901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5dff2-34a8-46ae-8ac3-7d754d1948e0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:47.397969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b0ed8-e1c8-4a3a-b573-782ff985b3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:47.397969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b0ed8-e1c8-4a3a-b573-782ff985b3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:47.397969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b0ed8-e1c8-4a3a-b573-782ff985b3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:47.397969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b0ed8-e1c8-4a3a-b573-782ff985b3b4 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:47.398433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b0ed8-e1c8-4a3a-b573-782ff985b3b4 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:47.399793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b0ed8-e1c8-4a3a-b573-782ff985b3b4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:47.401739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 111/221] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:47 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:47.413998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b0ed8-e1c8-4a3a-b573-782ff985b3b4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:47.524691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5dff2-34a8-46ae-8ac3-7d754d1948e0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:47.530947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b0ed8-e1c8-4a3a-b573-782ff985b3b4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:47.543355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:47.563788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037e621-905a-489f-bd19-42962ec1c3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:47.564429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037e621-905a-489f-bd19-42962ec1c3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:47.564798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037e621-905a-489f-bd19-42962ec1c3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:47.565072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037e621-905a-489f-bd19-42962ec1c3f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:47.573210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f6156-b167-42ef-83f4-ea4581662322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:47.573840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6156-b167-42ef-83f4-ea4581662322 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:47.574136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6156-b167-42ef-83f4-ea4581662322 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:47.574407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6156-b167-42ef-83f4-ea4581662322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:47.575131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f6156-b167-42ef-83f4-ea4581662322 None admin] RBAC: Adding query filter params (role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user.id=dcf2e6a2e4e34d41849ea6df285cd403) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:47.575438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f6156-b167-42ef-83f4-ea4581662322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:47.575809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037e621-905a-489f-bd19-42962ec1c3f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:47.576909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f6156-b167-42ef-83f4-ea4581662322 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3037e621-905a-489f-bd19-42962ec1c3f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:47.580875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:47.581727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:47.582541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:47.583130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 112/223] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:47 2022] GET /identity/v3/role_assignments?user.id=dcf2e6a2e4e34d41849ea6df285cd403&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=d36487377a664531a7f2e06f2f648a6f => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:47.586862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 111/223] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:47.587843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037e621-905a-489f-bd19-42962ec1c3f3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:47.595529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88178455-5f10-49d1-a2ec-81bf94f8e686 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:47.595529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88178455-5f10-49d1-a2ec-81bf94f8e686 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:47.595529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88178455-5f10-49d1-a2ec-81bf94f8e686 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:47.595529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 113/224] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:47 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:47.600967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a9133-d1a9-488f-9627-1d77e2f59dd9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:47.601249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a9133-d1a9-488f-9627-1d77e2f59dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:47.601510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a9133-d1a9-488f-9627-1d77e2f59dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:47.635636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-bf5a9133-d1a9-488f-9627-1d77e2f59dd9 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:47.636761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bf5a9133-d1a9-488f-9627-1d77e2f59dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:47.666737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:47.763558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 114/225] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:13:47.783650 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037e621-905a-489f-bd19-42962ec1c3f3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:47.820137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c832da-d4da-4b50-9972-ba36a1a45d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:47.820855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c832da-d4da-4b50-9972-ba36a1a45d7b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:47.821137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c832da-d4da-4b50-9972-ba36a1a45d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:47.821407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c832da-d4da-4b50-9972-ba36a1a45d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:47.822104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c832da-d4da-4b50-9972-ba36a1a45d7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:47.822409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c832da-d4da-4b50-9972-ba36a1a45d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:47.823784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c832da-d4da-4b50-9972-ba36a1a45d7b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:47.832336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 112/226] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:47.837839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a9133-d1a9-488f-9627-1d77e2f59dd9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:47.975105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a9133-d1a9-488f-9627-1d77e2f59dd9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:47.976514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:48.015512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81614-ede4-48ff-b9c6-1d8d7065f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:48.016167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81614-ede4-48ff-b9c6-1d8d7065f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:48.016444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81614-ede4-48ff-b9c6-1d8d7065f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:48.016714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81614-ede4-48ff-b9c6-1d8d7065f9f0 None admin] PATH_INFO: `/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:48.017889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81614-ede4-48ff-b9c6-1d8d7065f9f0 None admin] RBAC: Authorizing `identity:get_user(user_id=a36e7ae1cb5d4a3f9c474d465c436e6d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:48.019374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81614-ede4-48ff-b9c6-1d8d7065f9f0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:48.021265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 115/227] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:47 2022] GET /identity/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:48.026785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c832da-d4da-4b50-9972-ba36a1a45d7b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:48.145977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:48.150516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c832da-d4da-4b50-9972-ba36a1a45d7b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:48.196598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a555a-5f10-4324-930c-1dd1ce10da20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:48.197592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a555a-5f10-4324-930c-1dd1ce10da20 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:48.198110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a555a-5f10-4324-930c-1dd1ce10da20 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:48.198621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a555a-5f10-4324-930c-1dd1ce10da20 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:48.202514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a555a-5f10-4324-930c-1dd1ce10da20 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:48.204235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a555a-5f10-4324-930c-1dd1ce10da20 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:48.206390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 113/228] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:48 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:48.211955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81614-ede4-48ff-b9c6-1d8d7065f9f0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:48.230900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a555a-5f10-4324-930c-1dd1ce10da20 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:48.392450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81614-ede4-48ff-b9c6-1d8d7065f9f0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:48.410778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a555a-5f10-4324-930c-1dd1ce10da20 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:48.465242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07388597-0e5b-4008-9b72-bf64bdffd0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:48.465913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07388597-0e5b-4008-9b72-bf64bdffd0e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:48.466211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07388597-0e5b-4008-9b72-bf64bdffd0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:48.466479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07388597-0e5b-4008-9b72-bf64bdffd0e1 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/users/a36e7ae1cb5d4a3f9c474d465c436e6d/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:48.468613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6c2ca-f452-465f-9f0b-e7fc2221faa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:48.469248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6c2ca-f452-465f-9f0b-e7fc2221faa5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:48.469519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6c2ca-f452-465f-9f0b-e7fc2221faa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:48.469787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6c2ca-f452-465f-9f0b-e7fc2221faa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:48.473655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07388597-0e5b-4008-9b72-bf64bdffd0e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user_id=a36e7ae1cb5d4a3f9c474d465c436e6d, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:48.475150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07388597-0e5b-4008-9b72-bf64bdffd0e1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:48.487628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6c2ca-f452-465f-9f0b-e7fc2221faa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-dbb6c2ca-f452-465f-9f0b-e7fc2221faa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:48.493624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:48.494850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:48.495727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 114/229] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:13:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:48.507926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 116/230] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:13:48 2022] PUT /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/users/a36e7ae1cb5d4a3f9c474d465c436e6d/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:13:48.508736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6c2ca-f452-465f-9f0b-e7fc2221faa5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:48.542061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:48.639695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6c2ca-f452-465f-9f0b-e7fc2221faa5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:48.677443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d05671-04e7-4bc6-ba20-d982414c132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:48.678172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d05671-04e7-4bc6-ba20-d982414c132d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:48.678453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d05671-04e7-4bc6-ba20-d982414c132d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:48.678724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d05671-04e7-4bc6-ba20-d982414c132d None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:48.679401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d05671-04e7-4bc6-ba20-d982414c132d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:48.679703 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d05671-04e7-4bc6-ba20-d982414c132d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:48.681223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d05671-04e7-4bc6-ba20-d982414c132d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:48.691587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 115/231] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:13:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:48.697349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07388597-0e5b-4008-9b72-bf64bdffd0e1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:48.737336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:48.814612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07388597-0e5b-4008-9b72-bf64bdffd0e1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:48.859320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bfee9-e62e-4f9e-9989-8fa7aead7430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:48.859962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bfee9-e62e-4f9e-9989-8fa7aead7430 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:48.860240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bfee9-e62e-4f9e-9989-8fa7aead7430 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:48.860510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bfee9-e62e-4f9e-9989-8fa7aead7430 None admin] PATH_INFO: `/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:48.861701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bfee9-e62e-4f9e-9989-8fa7aead7430 None admin] RBAC: Authorizing `identity:get_user(user_id=a36e7ae1cb5d4a3f9c474d465c436e6d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:48.863482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bfee9-e62e-4f9e-9989-8fa7aead7430 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:48.865178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 117/232] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:48 2022] GET /identity/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:48.870905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d05671-04e7-4bc6-ba20-d982414c132d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:48.970756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:48.981334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d05671-04e7-4bc6-ba20-d982414c132d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:49.027136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc9d52-128c-4ec9-8c31-80c005d5357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:49.027911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc9d52-128c-4ec9-8c31-80c005d5357b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:49.028114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc9d52-128c-4ec9-8c31-80c005d5357b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:49.028396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc9d52-128c-4ec9-8c31-80c005d5357b None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:49.029597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc9d52-128c-4ec9-8c31-80c005d5357b None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:49.031077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc9d52-128c-4ec9-8c31-80c005d5357b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:49.032696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 116/233] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:48 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:49.038661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bfee9-e62e-4f9e-9989-8fa7aead7430 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:49.167252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:49.172159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bfee9-e62e-4f9e-9989-8fa7aead7430 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:49.215026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279ed7d-2f8e-4684-9643-9f554ea755ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:49.215691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279ed7d-2f8e-4684-9643-9f554ea755ce None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:49.215969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279ed7d-2f8e-4684-9643-9f554ea755ce None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:49.216246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279ed7d-2f8e-4684-9643-9f554ea755ce None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/users/a36e7ae1cb5d4a3f9c474d465c436e6d/roles/9578cd8bc0a8498f99757fd0111e48e8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:49.219023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279ed7d-2f8e-4684-9643-9f554ea755ce None admin] RBAC: Authorizing `identity:create_grant(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user_id=a36e7ae1cb5d4a3f9c474d465c436e6d, role_id=9578cd8bc0a8498f99757fd0111e48e8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:49.220513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279ed7d-2f8e-4684-9643-9f554ea755ce None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:49.242098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 118/234] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:13:49 2022] PUT /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/users/a36e7ae1cb5d4a3f9c474d465c436e6d/roles/9578cd8bc0a8498f99757fd0111e48e8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:13:49.474711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64525d-6ca6-4227-bba1-9dfa4503a046 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:49.475062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64525d-6ca6-4227-bba1-9dfa4503a046 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:49.475230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64525d-6ca6-4227-bba1-9dfa4503a046 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:49.476834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 117/235] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:49.483480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e66f9e-56fd-4441-894d-88e88b78a737 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:49.484030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e66f9e-56fd-4441-894d-88e88b78a737 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:49.484196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e66f9e-56fd-4441-894d-88e88b78a737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:49.504036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f8e66f9e-56fd-4441-894d-88e88b78a737 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:49.505248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f8e66f9e-56fd-4441-894d-88e88b78a737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:49.591608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 119/236] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:49.643329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0f589-2ae2-41a8-97ee-547e7aa605b5 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:49.643959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0f589-2ae2-41a8-97ee-547e7aa605b5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:49.644467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0f589-2ae2-41a8-97ee-547e7aa605b5 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:49.646294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 118/237] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:49.651733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed7586-011c-4661-a10b-93d26dfd2ae4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:49.652276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed7586-011c-4661-a10b-93d26dfd2ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:49.652766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed7586-011c-4661-a10b-93d26dfd2ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:49.671887 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-34ed7586-011c-4661-a10b-93d26dfd2ae4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:49.673419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-34ed7586-011c-4661-a10b-93d26dfd2ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:49.760307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 120/238] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:49.990669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0f589-2ae2-41a8-97ee-547e7aa605b5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:50.083208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed7586-011c-4661-a10b-93d26dfd2ae4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:50.105604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0f589-2ae2-41a8-97ee-547e7aa605b5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:50.126623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:50.178069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a63a9-920e-4cc3-8d3d-d216edb758c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:50.178702 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a63a9-920e-4cc3-8d3d-d216edb758c4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:50.178979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a63a9-920e-4cc3-8d3d-d216edb758c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:50.179252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a63a9-920e-4cc3-8d3d-d216edb758c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:50.180897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a63a9-920e-4cc3-8d3d-d216edb758c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:50.182358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a63a9-920e-4cc3-8d3d-d216edb758c4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:50.183913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 119/239] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:50.196719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a63a9-920e-4cc3-8d3d-d216edb758c4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:50.256136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed7586-011c-4661-a10b-93d26dfd2ae4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:50.271113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:50.297017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e7a6c-8d09-45a8-a563-573148830e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:50.297655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e7a6c-8d09-45a8-a563-573148830e7d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:50.297957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e7a6c-8d09-45a8-a563-573148830e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:50.298227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e7a6c-8d09-45a8-a563-573148830e7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:50.309327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e7a6c-8d09-45a8-a563-573148830e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d72e7a6c-8d09-45a8-a563-573148830e7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:50.315353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:50.316208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:50.316990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 121/240] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:13:50.318519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da62ee1-2207-485c-bd6c-af74c268eae0 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:50.319024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da62ee1-2207-485c-bd6c-af74c268eae0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:50.319427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da62ee1-2207-485c-bd6c-af74c268eae0 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:50.321048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 122/241] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:50.330375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da62ee1-2207-485c-bd6c-af74c268eae0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:50.355403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a63a9-920e-4cc3-8d3d-d216edb758c4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:50.401663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75c29b-0db0-462d-ae5d-5e35d42fe41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:50.402885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75c29b-0db0-462d-ae5d-5e35d42fe41e None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:50.403261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75c29b-0db0-462d-ae5d-5e35d42fe41e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:50.403844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75c29b-0db0-462d-ae5d-5e35d42fe41e None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:50.404601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75c29b-0db0-462d-ae5d-5e35d42fe41e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:50.406216 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75c29b-0db0-462d-ae5d-5e35d42fe41e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:50.422980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:50.441463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 120/242] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:13:50 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:50.443078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26b7a-d6cb-4dd2-a03f-bd582c96d68d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:50.443451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26b7a-d6cb-4dd2-a03f-bd582c96d68d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:50.443880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26b7a-d6cb-4dd2-a03f-bd582c96d68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:50.455098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da62ee1-2207-485c-bd6c-af74c268eae0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:50.473560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-67b26b7a-d6cb-4dd2-a03f-bd582c96d68d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:50.474826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-67b26b7a-d6cb-4dd2-a03f-bd582c96d68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:50.500939 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175318bf-673e-4182-b68b-9d68849d6f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:50.503191 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175318bf-673e-4182-b68b-9d68849d6f3d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:50.503734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175318bf-673e-4182-b68b-9d68849d6f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:50.504231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175318bf-673e-4182-b68b-9d68849d6f3d None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:50.505714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175318bf-673e-4182-b68b-9d68849d6f3d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:50.506589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175318bf-673e-4182-b68b-9d68849d6f3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:50.509393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175318bf-673e-4182-b68b-9d68849d6f3d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:50.528567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 123/243] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:50.536317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175318bf-673e-4182-b68b-9d68849d6f3d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:50.628727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 121/244] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:50.690798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175318bf-673e-4182-b68b-9d68849d6f3d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:50.726880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7a9e2-d56a-4ace-b731-e823c28af9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:50.727853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7a9e2-d56a-4ace-b731-e823c28af9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:50.728331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7a9e2-d56a-4ace-b731-e823c28af9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:50.728822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7a9e2-d56a-4ace-b731-e823c28af9d6 None admin] PATH_INFO: `/v3/users/2bee92248caf432b9dec0079a72e029a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:50.743519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:50.746602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7a9e2-d56a-4ace-b731-e823c28af9d6 None admin] RBAC: Authorizing `identity:get_user(user_id=2bee92248caf432b9dec0079a72e029a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:50.749805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7a9e2-d56a-4ace-b731-e823c28af9d6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:50.752424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 124/245] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:50 2022] GET /identity/v3/users/2bee92248caf432b9dec0079a72e029a => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:50.758035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b26b7a-d6cb-4dd2-a03f-bd582c96d68d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:50.775413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:50.887763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b26b7a-d6cb-4dd2-a03f-bd582c96d68d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:50.928285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7131827-d1ea-4d4e-9259-ccfac199fc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:50.929262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7131827-d1ea-4d4e-9259-ccfac199fc65 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:50.929742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7131827-d1ea-4d4e-9259-ccfac199fc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:50.930248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7131827-d1ea-4d4e-9259-ccfac199fc65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:50.941606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7a9e2-d56a-4ace-b731-e823c28af9d6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:51.010630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7131827-d1ea-4d4e-9259-ccfac199fc65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:51.012594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7131827-d1ea-4d4e-9259-ccfac199fc65 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:51.033855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:51.048393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7a9e2-d56a-4ace-b731-e823c28af9d6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:51.095148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa50644-7860-4f3c-813e-882b19b0174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:51.095148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa50644-7860-4f3c-813e-882b19b0174a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.095427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa50644-7860-4f3c-813e-882b19b0174a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.095675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa50644-7860-4f3c-813e-882b19b0174a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:51.112466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa50644-7860-4f3c-813e-882b19b0174a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-faa50644-7860-4f3c-813e-882b19b0174a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:51.117696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:51.118597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:51.119382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:51.120482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 125/246] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:51.125543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa50644-7860-4f3c-813e-882b19b0174a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:51.151509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 122/247] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:13:51.155971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7131827-d1ea-4d4e-9259-ccfac199fc65 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:51.208347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:51.269185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa50644-7860-4f3c-813e-882b19b0174a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:51.294381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7131827-d1ea-4d4e-9259-ccfac199fc65 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:51.320911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:51.327863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd5fa9-f900-4635-b918-a8e9d351f4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:51.328824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd5fa9-f900-4635-b918-a8e9d351f4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.329302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd5fa9-f900-4635-b918-a8e9d351f4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.331364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd5fa9-f900-4635-b918-a8e9d351f4f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:51.332272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd5fa9-f900-4635-b918-a8e9d351f4f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:51.332857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd5fa9-f900-4635-b918-a8e9d351f4f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:51.335285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd5fa9-f900-4635-b918-a8e9d351f4f4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:51.354979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 126/248] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:51.361139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd5fa9-f900-4635-b918-a8e9d351f4f4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:51.364452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721a912-47fb-4f6c-83ef-b17b56c09107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:51.365089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721a912-47fb-4f6c-83ef-b17b56c09107 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.365366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721a912-47fb-4f6c-83ef-b17b56c09107 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.365633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721a912-47fb-4f6c-83ef-b17b56c09107 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:51.366842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721a912-47fb-4f6c-83ef-b17b56c09107 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:51.368635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721a912-47fb-4f6c-83ef-b17b56c09107 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:51.370183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 123/249] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:51 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:51.382832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721a912-47fb-4f6c-83ef-b17b56c09107 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:51.567623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd5fa9-f900-4635-b918-a8e9d351f4f4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:51.580301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721a912-47fb-4f6c-83ef-b17b56c09107 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:51.611538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:51.629079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d33194-207a-41b1-a81e-b8c1ca1041e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:51.629920 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d33194-207a-41b1-a81e-b8c1ca1041e7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.630360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d33194-207a-41b1-a81e-b8c1ca1041e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.630737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d33194-207a-41b1-a81e-b8c1ca1041e7 None admin] PATH_INFO: `/v3/users/dcf2e6a2e4e34d41849ea6df285cd403` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:51.632239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d33194-207a-41b1-a81e-b8c1ca1041e7 None admin] RBAC: Authorizing `identity:get_user(user_id=dcf2e6a2e4e34d41849ea6df285cd403)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:51.633838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d33194-207a-41b1-a81e-b8c1ca1041e7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:51.636212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 127/250] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:51 2022] GET /identity/v3/users/dcf2e6a2e4e34d41849ea6df285cd403 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:51.642324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d33194-207a-41b1-a81e-b8c1ca1041e7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:51.667787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd709fdc-89da-4e9a-864c-590d183b083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:51.668469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd709fdc-89da-4e9a-864c-590d183b083b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.668683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd709fdc-89da-4e9a-864c-590d183b083b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.668956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd709fdc-89da-4e9a-864c-590d183b083b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:51.669693 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd709fdc-89da-4e9a-864c-590d183b083b None admin] RBAC: Adding query filter params (role.id=439053f3a99d43f5abe2fb7dec39df9d, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user.id=2bee92248caf432b9dec0079a72e029a) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:51.670066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd709fdc-89da-4e9a-864c-590d183b083b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:51.676527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd709fdc-89da-4e9a-864c-590d183b083b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:51.691629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 124/251] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:51 2022] GET /identity/v3/role_assignments?user.id=2bee92248caf432b9dec0079a72e029a&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=439053f3a99d43f5abe2fb7dec39df9d => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:51.777915 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dfbda-7bc0-4b03-b081-07173cf1f379 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.778248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dfbda-7bc0-4b03-b081-07173cf1f379 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.778488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dfbda-7bc0-4b03-b081-07173cf1f379 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:51.780203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 125/252] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:51.785584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703a4ae-eb99-41fb-a107-fc7edde705fc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.785765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d33194-207a-41b1-a81e-b8c1ca1041e7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:51.786010 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703a4ae-eb99-41fb-a107-fc7edde705fc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.786168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703a4ae-eb99-41fb-a107-fc7edde705fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:51.827803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0f224-422f-4146-ad67-9bac8158931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:51.828384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0f224-422f-4146-ad67-9bac8158931e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.828670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0f224-422f-4146-ad67-9bac8158931e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.828944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0f224-422f-4146-ad67-9bac8158931e None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:51.830227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0f224-422f-4146-ad67-9bac8158931e None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:51.831024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6703a4ae-eb99-41fb-a107-fc7edde705fc None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:51.831692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0f224-422f-4146-ad67-9bac8158931e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:51.832246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6703a4ae-eb99-41fb-a107-fc7edde705fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:51.833456 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 128/253] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:51 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:51.840615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0f224-422f-4146-ad67-9bac8158931e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:51.910284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:51.934290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:51.964748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0f224-422f-4146-ad67-9bac8158931e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:51.968764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 126/254] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:51.970258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a330c01-dca0-4d43-8dab-062bed46e0ec None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.970541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a330c01-dca0-4d43-8dab-062bed46e0ec None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.970811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a330c01-dca0-4d43-8dab-062bed46e0ec None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:51.972275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 127/255] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:51.987322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9e7bc-8f0f-43e4-b5c0-0f7f4ead608c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:51.987590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9e7bc-8f0f-43e4-b5c0-0f7f4ead608c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:51.988118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9e7bc-8f0f-43e4-b5c0-0f7f4ead608c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:52.002715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798df066-4422-40b1-93b1-05ea0aef4755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:52.003395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798df066-4422-40b1-93b1-05ea0aef4755 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:52.003676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798df066-4422-40b1-93b1-05ea0aef4755 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:52.003951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798df066-4422-40b1-93b1-05ea0aef4755 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/users/dcf2e6a2e4e34d41849ea6df285cd403/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:52.005918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798df066-4422-40b1-93b1-05ea0aef4755 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user_id=dcf2e6a2e4e34d41849ea6df285cd403, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:52.008281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798df066-4422-40b1-93b1-05ea0aef4755 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:52.030254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 129/256] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:13:51 2022] PUT /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/users/dcf2e6a2e4e34d41849ea6df285cd403/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:13:52.034074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-03b9e7bc-8f0f-43e4-b5c0-0f7f4ead608c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:52.035283 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-03b9e7bc-8f0f-43e4-b5c0-0f7f4ead608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:52.144568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 128/257] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:52.399651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798df066-4422-40b1-93b1-05ea0aef4755 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:52.510737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798df066-4422-40b1-93b1-05ea0aef4755 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:52.528471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:52.547570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a794258-eaaa-411b-9d55-ca798e5197a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:52.548205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a794258-eaaa-411b-9d55-ca798e5197a9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:52.548494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a794258-eaaa-411b-9d55-ca798e5197a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:52.548945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a794258-eaaa-411b-9d55-ca798e5197a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:52.559972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a794258-eaaa-411b-9d55-ca798e5197a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-0a794258-eaaa-411b-9d55-ca798e5197a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:52.565732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:52.566676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:13:52.567447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:52.571494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 130/258] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:52.573357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9e7bc-8f0f-43e4-b5c0-0f7f4ead608c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:52.633616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a794258-eaaa-411b-9d55-ca798e5197a9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:52.784350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a794258-eaaa-411b-9d55-ca798e5197a9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:52.799808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9e7bc-8f0f-43e4-b5c0-0f7f4ead608c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:52.807814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:52.853832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d233c35-6c74-4a6d-8695-0b4b1884d504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:52.854905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d233c35-6c74-4a6d-8695-0b4b1884d504 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:52.855379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d233c35-6c74-4a6d-8695-0b4b1884d504 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:52.855861 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d233c35-6c74-4a6d-8695-0b4b1884d504 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:52.856721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d233c35-6c74-4a6d-8695-0b4b1884d504 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:52.857214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d233c35-6c74-4a6d-8695-0b4b1884d504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:52.861090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d233c35-6c74-4a6d-8695-0b4b1884d504 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:52.870553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 129/259] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:52.874463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d8368-9fd4-4769-af00-fb830593e782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:52.876364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d233c35-6c74-4a6d-8695-0b4b1884d504 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:52.885564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d8368-9fd4-4769-af00-fb830593e782 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:52.886194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d8368-9fd4-4769-af00-fb830593e782 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:52.886689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d8368-9fd4-4769-af00-fb830593e782 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:52.898545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d8368-9fd4-4769-af00-fb830593e782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5a3d8368-9fd4-4769-af00-fb830593e782 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:52.905420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:52.906301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:13:52.907062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:52.910231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 131/260] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:13:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:13:52.919318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d8368-9fd4-4769-af00-fb830593e782 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:52.972021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:53.085345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d8368-9fd4-4769-af00-fb830593e782 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:53.127947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf551219-eb59-4cb0-a9db-c2ffeb8bbccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:53.128564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf551219-eb59-4cb0-a9db-c2ffeb8bbccd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:53.128844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf551219-eb59-4cb0-a9db-c2ffeb8bbccd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:53.129117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf551219-eb59-4cb0-a9db-c2ffeb8bbccd None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:53.129775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf551219-eb59-4cb0-a9db-c2ffeb8bbccd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:53.130101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf551219-eb59-4cb0-a9db-c2ffeb8bbccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:53.132002 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf551219-eb59-4cb0-a9db-c2ffeb8bbccd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:53.145125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 132/261] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:13:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:53.148528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d233c35-6c74-4a6d-8695-0b4b1884d504 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:53.151011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf551219-eb59-4cb0-a9db-c2ffeb8bbccd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:53.220209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f21a75-0a69-4847-bf0e-357345d237da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:53.221215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f21a75-0a69-4847-bf0e-357345d237da None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:53.221708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f21a75-0a69-4847-bf0e-357345d237da None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:53.222325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f21a75-0a69-4847-bf0e-357345d237da None admin] PATH_INFO: `/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:53.224468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f21a75-0a69-4847-bf0e-357345d237da None admin] RBAC: Authorizing `identity:get_user(user_id=a36e7ae1cb5d4a3f9c474d465c436e6d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:53.230354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f21a75-0a69-4847-bf0e-357345d237da None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:53.233046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 130/262] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:52 2022] GET /identity/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d => generated 283 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:53.240015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f21a75-0a69-4847-bf0e-357345d237da None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:53.305109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:53.355535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:53.362322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf551219-eb59-4cb0-a9db-c2ffeb8bbccd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:53.414647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f21a75-0a69-4847-bf0e-357345d237da None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:53.450939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00beedc-6fce-4d5c-9271-2298327c5670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:53.452497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00beedc-6fce-4d5c-9271-2298327c5670 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:53.452973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00beedc-6fce-4d5c-9271-2298327c5670 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:53.453462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00beedc-6fce-4d5c-9271-2298327c5670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:53.456684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682552-085d-4fa7-a0e0-90402ccbab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:53.457507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682552-085d-4fa7-a0e0-90402ccbab92 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:53.458036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682552-085d-4fa7-a0e0-90402ccbab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:53.458520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682552-085d-4fa7-a0e0-90402ccbab92 None admin] PATH_INFO: `/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:53.461077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74682552-085d-4fa7-a0e0-90402ccbab92 None admin] RBAC: Authorizing `identity:get_user(user_id=a36e7ae1cb5d4a3f9c474d465c436e6d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:53.462738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74682552-085d-4fa7-a0e0-90402ccbab92 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:53.464803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 133/263] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:53 2022] GET /identity/v3/users/a36e7ae1cb5d4a3f9c474d465c436e6d => generated 283 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:53.470842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682552-085d-4fa7-a0e0-90402ccbab92 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:53.541507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00beedc-6fce-4d5c-9271-2298327c5670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:53.543961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00beedc-6fce-4d5c-9271-2298327c5670 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:53.644254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:53.743240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682552-085d-4fa7-a0e0-90402ccbab92 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:53.800613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 131/264] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 05:13:53.807125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00beedc-6fce-4d5c-9271-2298327c5670 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:53.833959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efd53d-6b4b-42ab-8093-a62fd7827fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:53.834993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efd53d-6b4b-42ab-8093-a62fd7827fae None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:53.835474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efd53d-6b4b-42ab-8093-a62fd7827fae None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:53.835943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efd53d-6b4b-42ab-8093-a62fd7827fae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:53.913319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:53.978323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efd53d-6b4b-42ab-8093-a62fd7827fae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:53.979782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efd53d-6b4b-42ab-8093-a62fd7827fae None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:53.991871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00beedc-6fce-4d5c-9271-2298327c5670 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:54.050679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa97c84-953b-4a2a-b0f9-7e3d9d32e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:54.051322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa97c84-953b-4a2a-b0f9-7e3d9d32e7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:54.051599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa97c84-953b-4a2a-b0f9-7e3d9d32e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:54.051867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa97c84-953b-4a2a-b0f9-7e3d9d32e7e4 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:54.053524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa97c84-953b-4a2a-b0f9-7e3d9d32e7e4 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:54.055004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa97c84-953b-4a2a-b0f9-7e3d9d32e7e4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:54.062093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 132/265] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:53 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:54.063587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c13a4-3ed2-4d57-a15e-447f7d7f58d3 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:54.064216 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c13a4-3ed2-4d57-a15e-447f7d7f58d3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:54.064770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c13a4-3ed2-4d57-a15e-447f7d7f58d3 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:54.066383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 133/266] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:54.067667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c13a4-3ed2-4d57-a15e-447f7d7f58d3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:54.078242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:54.177070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 134/267] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 05:13:54.178271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360700-4809-4760-89b2-11d01667e045 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:54.178556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360700-4809-4760-89b2-11d01667e045 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:54.178829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360700-4809-4760-89b2-11d01667e045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:54.213849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b6360700-4809-4760-89b2-11d01667e045 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:54.217435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b6360700-4809-4760-89b2-11d01667e045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:54.263821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c13a4-3ed2-4d57-a15e-447f7d7f58d3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:54.301699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:54.304588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d1cd6-baa3-432b-9df4-fa9a26de62ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:54.305390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d1cd6-baa3-432b-9df4-fa9a26de62ee None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:54.306452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d1cd6-baa3-432b-9df4-fa9a26de62ee None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:54.306452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d1cd6-baa3-432b-9df4-fa9a26de62ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:54.307172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d1cd6-baa3-432b-9df4-fa9a26de62ee None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user.id=a36e7ae1cb5d4a3f9c474d465c436e6d) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:54.307874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d1cd6-baa3-432b-9df4-fa9a26de62ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:54.309950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d1cd6-baa3-432b-9df4-fa9a26de62ee None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:54.311489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 135/268] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:54.312382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6360700-4809-4760-89b2-11d01667e045 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:54.317466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 134/269] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:54 2022] GET /identity/v3/role_assignments?user.id=a36e7ae1cb5d4a3f9c474d465c436e6d&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:54.318974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f4ade-8057-431d-bb69-2462088d9ca5 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:54.319473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f4ade-8057-431d-bb69-2462088d9ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:54.319985 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f4ade-8057-431d-bb69-2462088d9ca5 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:54.321640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 135/270] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:54.327757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372f1ef-2d86-4b46-9b60-7a76bbd7e649 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:54.328310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372f1ef-2d86-4b46-9b60-7a76bbd7e649 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:54.328808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372f1ef-2d86-4b46-9b60-7a76bbd7e649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:54.347916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0372f1ef-2d86-4b46-9b60-7a76bbd7e649 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:54.349406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0372f1ef-2d86-4b46-9b60-7a76bbd7e649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:54.416473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:54.425911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 136/271] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:54.436450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6360700-4809-4760-89b2-11d01667e045 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:54.474809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ee5a9-6728-4113-a976-ef3e97b8c335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:54.475450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ee5a9-6728-4113-a976-ef3e97b8c335 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:54.475727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ee5a9-6728-4113-a976-ef3e97b8c335 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:54.475997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ee5a9-6728-4113-a976-ef3e97b8c335 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:54.477182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ee5a9-6728-4113-a976-ef3e97b8c335 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:54.478676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ee5a9-6728-4113-a976-ef3e97b8c335 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:54.480347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 136/272] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:54 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:54.488028 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372f1ef-2d86-4b46-9b60-7a76bbd7e649 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:54.630259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372f1ef-2d86-4b46-9b60-7a76bbd7e649 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:54.673960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46217-5584-4766-b174-6ea5ea1cbc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:54.674597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46217-5584-4766-b174-6ea5ea1cbc6c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:54.674824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46217-5584-4766-b174-6ea5ea1cbc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:54.675097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46217-5584-4766-b174-6ea5ea1cbc6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:54.675824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46217-5584-4766-b174-6ea5ea1cbc6c None admin] RBAC: Adding query filter params (role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user.id=a36e7ae1cb5d4a3f9c474d465c436e6d) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:54.676130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46217-5584-4766-b174-6ea5ea1cbc6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:54.677570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46217-5584-4766-b174-6ea5ea1cbc6c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:54.683907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 137/273] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:54 2022] GET /identity/v3/role_assignments?user.id=a36e7ae1cb5d4a3f9c474d465c436e6d&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:54.691189 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ee5a9-6728-4113-a976-ef3e97b8c335 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:54.766916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:54.810813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ee5a9-6728-4113-a976-ef3e97b8c335 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:54.849266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71830c9e-7ae3-457f-a4cc-1210c2d712fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:54.849947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71830c9e-7ae3-457f-a4cc-1210c2d712fa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:54.850239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71830c9e-7ae3-457f-a4cc-1210c2d712fa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:54.850508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71830c9e-7ae3-457f-a4cc-1210c2d712fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:54.863062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71830c9e-7ae3-457f-a4cc-1210c2d712fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-71830c9e-7ae3-457f-a4cc-1210c2d712fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:54.868119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:54.869003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:54.869791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 137/274] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:54.876469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46217-5584-4766-b174-6ea5ea1cbc6c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:54.893372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71830c9e-7ae3-457f-a4cc-1210c2d712fa None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:54.965362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:55.017105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71830c9e-7ae3-457f-a4cc-1210c2d712fa None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:55.019081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46217-5584-4766-b174-6ea5ea1cbc6c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:55.044126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:55.060900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95176fc-08ca-40db-ac27-5d803bda84ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:55.061860 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95176fc-08ca-40db-ac27-5d803bda84ae None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.062172 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95176fc-08ca-40db-ac27-5d803bda84ae None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.062442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95176fc-08ca-40db-ac27-5d803bda84ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.065197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95176fc-08ca-40db-ac27-5d803bda84ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:55.065627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b1ea3-c247-463c-9657-704faa2327db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:55.066485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b1ea3-c247-463c-9657-704faa2327db None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.066764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95176fc-08ca-40db-ac27-5d803bda84ae None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:55.067154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b1ea3-c247-463c-9657-704faa2327db None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.067603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b1ea3-c247-463c-9657-704faa2327db None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.068473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 138/275] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:55.073058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b1ea3-c247-463c-9657-704faa2327db None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:55.073058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b1ea3-c247-463c-9657-704faa2327db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:55.073058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b1ea3-c247-463c-9657-704faa2327db None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:55.075113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95176fc-08ca-40db-ac27-5d803bda84ae None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:55.090261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 138/276] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:55.095776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b1ea3-c247-463c-9657-704faa2327db None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:55.217569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95176fc-08ca-40db-ac27-5d803bda84ae None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:55.229746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b1ea3-c247-463c-9657-704faa2327db None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:55.257913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:55.259780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f5b00-9c7e-4efa-a5c9-6cdac487209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:55.260464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f5b00-9c7e-4efa-a5c9-6cdac487209b None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.260739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f5b00-9c7e-4efa-a5c9-6cdac487209b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.261009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f5b00-9c7e-4efa-a5c9-6cdac487209b None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.261653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f5b00-9c7e-4efa-a5c9-6cdac487209b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:55.263105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f5b00-9c7e-4efa-a5c9-6cdac487209b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:55.270366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb5199-f157-4ade-a210-22b44d30fa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:55.270991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb5199-f157-4ade-a210-22b44d30fa50 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.271273 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb5199-f157-4ade-a210-22b44d30fa50 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.271553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb5199-f157-4ade-a210-22b44d30fa50 None admin] PATH_INFO: `/v3/users/cbbadad7e843449ab937a4b0e5f0a6bc` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.287698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 139/277] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:13:55 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:55.287870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b4905-1a97-457c-a168-9eb692c85ecd None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.288206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b4905-1a97-457c-a168-9eb692c85ecd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.288303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb5199-f157-4ade-a210-22b44d30fa50 None admin] RBAC: Authorizing `identity:get_user(user_id=cbbadad7e843449ab937a4b0e5f0a6bc)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:55.288374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b4905-1a97-457c-a168-9eb692c85ecd None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.298052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 140/278] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:55.298242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb5199-f157-4ade-a210-22b44d30fa50 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:55.298319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b4d3-e8ea-40ed-ab84-c0ed8c9769e8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.298319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b4d3-e8ea-40ed-ab84-c0ed8c9769e8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.298319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b4d3-e8ea-40ed-ab84-c0ed8c9769e8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.298538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 139/279] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:55 2022] GET /identity/v3/users/cbbadad7e843449ab937a4b0e5f0a6bc => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:55.298603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 141/280] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:55.298664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb5199-f157-4ade-a210-22b44d30fa50 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:55.298737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb898e32-d5aa-4678-805e-4875fcde1d64 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.299138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb898e32-d5aa-4678-805e-4875fcde1d64 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.299299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb898e32-d5aa-4678-805e-4875fcde1d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.323244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-bb898e32-d5aa-4678-805e-4875fcde1d64 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:55.324828 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bb898e32-d5aa-4678-805e-4875fcde1d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:55.382396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:55.409664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb5199-f157-4ade-a210-22b44d30fa50 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:55.412458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 142/281] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:55.413770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b839c-9076-4eca-9cb6-0d64f4801152 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.414083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b839c-9076-4eca-9cb6-0d64f4801152 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.414354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b839c-9076-4eca-9cb6-0d64f4801152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.441797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0a5b839c-9076-4eca-9cb6-0d64f4801152 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:55.443059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0a5b839c-9076-4eca-9cb6-0d64f4801152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:55.460184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d22657-53f4-4f9d-8fdf-da8f29862f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:55.461461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d22657-53f4-4f9d-8fdf-da8f29862f66 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.462366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d22657-53f4-4f9d-8fdf-da8f29862f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.463305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d22657-53f4-4f9d-8fdf-da8f29862f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.464478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d22657-53f4-4f9d-8fdf-da8f29862f66 None admin] RBAC: Adding query filter params (role.id=d36487377a664531a7f2e06f2f648a6f, user.id=cbbadad7e843449ab937a4b0e5f0a6bc, scope.system=all) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:55.465043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d22657-53f4-4f9d-8fdf-da8f29862f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:55.466969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d22657-53f4-4f9d-8fdf-da8f29862f66 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:55.473862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 140/282] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Sep 21 05:13:55 2022] GET /identity/v3/role_assignments?user.id=cbbadad7e843449ab937a4b0e5f0a6bc&scope.system=all&role.id=d36487377a664531a7f2e06f2f648a6f => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:55.534010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 143/283] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:55.691552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d22657-53f4-4f9d-8fdf-da8f29862f66 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:55.741606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:55.803677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d22657-53f4-4f9d-8fdf-da8f29862f66 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:55.822184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b839c-9076-4eca-9cb6-0d64f4801152 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:55.855172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04171613-6602-4e3f-a1d4-7043ee7b93a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:55.855809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04171613-6602-4e3f-a1d4-7043ee7b93a8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:55.856083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04171613-6602-4e3f-a1d4-7043ee7b93a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:55.856353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04171613-6602-4e3f-a1d4-7043ee7b93a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:55.874695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04171613-6602-4e3f-a1d4-7043ee7b93a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-04171613-6602-4e3f-a1d4-7043ee7b93a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:55.879528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:55.880367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:55.881143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 141/284] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:55.892286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04171613-6602-4e3f-a1d4-7043ee7b93a8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:56.005500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b839c-9076-4eca-9cb6-0d64f4801152 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:56.015761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:56.061919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a5da9-fd1e-4a78-82bc-270777657f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:56.062585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5da9-fd1e-4a78-82bc-270777657f3a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:56.062866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5da9-fd1e-4a78-82bc-270777657f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:56.063135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5da9-fd1e-4a78-82bc-270777657f3a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:56.072978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04171613-6602-4e3f-a1d4-7043ee7b93a8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:56.074844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:56.093707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a5da9-fd1e-4a78-82bc-270777657f3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-266a5da9-fd1e-4a78-82bc-270777657f3a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:56.108967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:56.110080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:56.111050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:56.113430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 144/285] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:56.119156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a5da9-fd1e-4a78-82bc-270777657f3a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:56.120626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab368f-15b3-4c69-968e-f492b2b53330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:56.121293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab368f-15b3-4c69-968e-f492b2b53330 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:56.121577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab368f-15b3-4c69-968e-f492b2b53330 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:56.121849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab368f-15b3-4c69-968e-f492b2b53330 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:56.122542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab368f-15b3-4c69-968e-f492b2b53330 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:56.122842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab368f-15b3-4c69-968e-f492b2b53330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:56.125640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab368f-15b3-4c69-968e-f492b2b53330 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:56.143857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 142/286] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:56.149583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab368f-15b3-4c69-968e-f492b2b53330 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:56.224329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a5da9-fd1e-4a78-82bc-270777657f3a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:56.268158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b409c-04d9-46f4-8067-df91782dce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:56.268989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b409c-04d9-46f4-8067-df91782dce7a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:56.269408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b409c-04d9-46f4-8067-df91782dce7a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:56.269824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b409c-04d9-46f4-8067-df91782dce7a None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:56.270648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b409c-04d9-46f4-8067-df91782dce7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:56.271086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b409c-04d9-46f4-8067-df91782dce7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:56.272605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b409c-04d9-46f4-8067-df91782dce7a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:56.281977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab368f-15b3-4c69-968e-f492b2b53330 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:56.295420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 145/287] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:56.300896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b409c-04d9-46f4-8067-df91782dce7a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:56.344720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe1876-f3a9-4421-88b1-5d9f6f512579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:56.345710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe1876-f3a9-4421-88b1-5d9f6f512579 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:56.346209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe1876-f3a9-4421-88b1-5d9f6f512579 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:56.346785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe1876-f3a9-4421-88b1-5d9f6f512579 None admin] PATH_INFO: `/v3/users/dcf2e6a2e4e34d41849ea6df285cd403` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:56.349194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe1876-f3a9-4421-88b1-5d9f6f512579 None admin] RBAC: Authorizing `identity:get_user(user_id=dcf2e6a2e4e34d41849ea6df285cd403)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:56.351855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe1876-f3a9-4421-88b1-5d9f6f512579 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:56.353829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 143/288] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:56 2022] GET /identity/v3/users/dcf2e6a2e4e34d41849ea6df285cd403 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:56.362395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe1876-f3a9-4421-88b1-5d9f6f512579 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:56.368904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:56.444321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:56.503373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b409c-04d9-46f4-8067-df91782dce7a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:56.569564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d095ba-d4ae-472e-944c-f5f6f770d0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:56.570141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d095ba-d4ae-472e-944c-f5f6f770d0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:56.570418 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d095ba-d4ae-472e-944c-f5f6f770d0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:56.570689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d095ba-d4ae-472e-944c-f5f6f770d0b4 None admin] PATH_INFO: `/v3/users/2bee92248caf432b9dec0079a72e029a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:56.573124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d095ba-d4ae-472e-944c-f5f6f770d0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=2bee92248caf432b9dec0079a72e029a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:56.574582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d095ba-d4ae-472e-944c-f5f6f770d0b4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:56.576753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 146/289] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:56 2022] GET /identity/v3/users/2bee92248caf432b9dec0079a72e029a => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:56.585570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d095ba-d4ae-472e-944c-f5f6f770d0b4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:56.591000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe1876-f3a9-4421-88b1-5d9f6f512579 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:56.673380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b31ba3-e520-4e46-9e79-37984c00a53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:56.674022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b31ba3-e520-4e46-9e79-37984c00a53c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:56.674305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b31ba3-e520-4e46-9e79-37984c00a53c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:56.674579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b31ba3-e520-4e46-9e79-37984c00a53c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:56.780185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:56.786061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d095ba-d4ae-472e-944c-f5f6f770d0b4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:56.802186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b31ba3-e520-4e46-9e79-37984c00a53c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:56.803689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b31ba3-e520-4e46-9e79-37984c00a53c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:56.846916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3e654-7f79-4117-8b91-f7625f3787f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:56.847563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3e654-7f79-4117-8b91-f7625f3787f0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:56.847834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3e654-7f79-4117-8b91-f7625f3787f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:56.848106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3e654-7f79-4117-8b91-f7625f3787f0 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:56.849234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3e654-7f79-4117-8b91-f7625f3787f0 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:56.850716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3e654-7f79-4117-8b91-f7625f3787f0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:56.853812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 147/290] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:56 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:56.859371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3e654-7f79-4117-8b91-f7625f3787f0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:56.874298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:56.989815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 144/291] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 05:13:56.995553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b31ba3-e520-4e46-9e79-37984c00a53c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:57.036289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3e654-7f79-4117-8b91-f7625f3787f0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:57.095388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b7713-7369-43fe-b722-1f8e6c5b0a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:57.096061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b7713-7369-43fe-b722-1f8e6c5b0a5e None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:57.096344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b7713-7369-43fe-b722-1f8e6c5b0a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:57.096612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b7713-7369-43fe-b722-1f8e6c5b0a5e None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/users/2bee92248caf432b9dec0079a72e029a/roles/439053f3a99d43f5abe2fb7dec39df9d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:57.099914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b7713-7369-43fe-b722-1f8e6c5b0a5e None admin] RBAC: Authorizing `identity:create_grant(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user_id=2bee92248caf432b9dec0079a72e029a, role_id=439053f3a99d43f5abe2fb7dec39df9d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:57.101310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b7713-7369-43fe-b722-1f8e6c5b0a5e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:57.130087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 148/292] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:13:56 2022] PUT /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/users/2bee92248caf432b9dec0079a72e029a/roles/439053f3a99d43f5abe2fb7dec39df9d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:13:57.154880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b31ba3-e520-4e46-9e79-37984c00a53c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:57.199279 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cb6af-9849-4a81-adae-ec18f8935f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:57.200374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cb6af-9849-4a81-adae-ec18f8935f1e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:57.200862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cb6af-9849-4a81-adae-ec18f8935f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:57.201867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cb6af-9849-4a81-adae-ec18f8935f1e None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:57.206394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cb6af-9849-4a81-adae-ec18f8935f1e None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:57.208146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cb6af-9849-4a81-adae-ec18f8935f1e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:57.210435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 145/293] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:13:56 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:13:57.217570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b7713-7369-43fe-b722-1f8e6c5b0a5e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:57.268585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:57.358961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b7713-7369-43fe-b722-1f8e6c5b0a5e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:57.394191 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7f872-60b3-4773-961c-23fb8f87475f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:57.394807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7f872-60b3-4773-961c-23fb8f87475f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:57.395089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7f872-60b3-4773-961c-23fb8f87475f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:57.395369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7f872-60b3-4773-961c-23fb8f87475f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:57.396097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7f872-60b3-4773-961c-23fb8f87475f None admin] RBAC: Adding query filter params (role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user.id=dcf2e6a2e4e34d41849ea6df285cd403) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:57.396407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7f872-60b3-4773-961c-23fb8f87475f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:57.397855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7f872-60b3-4773-961c-23fb8f87475f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:57.406405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 149/294] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:13:57 2022] GET /identity/v3/role_assignments?user.id=dcf2e6a2e4e34d41849ea6df285cd403&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=d36487377a664531a7f2e06f2f648a6f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:57.917114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dee41-70ef-4d51-b5c7-ec5d05e76a2f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:57.918001 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dee41-70ef-4d51-b5c7-ec5d05e76a2f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:57.918517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dee41-70ef-4d51-b5c7-ec5d05e76a2f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:57.920897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 146/295] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:57.926019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6066b3c-2749-4f85-9e4d-958630f59661 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:57.926869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6066b3c-2749-4f85-9e4d-958630f59661 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:57.927370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6066b3c-2749-4f85-9e4d-958630f59661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:57.949624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a6066b3c-2749-4f85-9e4d-958630f59661 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:57.951230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a6066b3c-2749-4f85-9e4d-958630f59661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:58.015281 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09001443-68f2-48d0-9605-2061e79e2ac7 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.015941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09001443-68f2-48d0-9605-2061e79e2ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.016507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09001443-68f2-48d0-9605-2061e79e2ac7 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.020923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 147/296] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:13:58.024318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60af9f-8a50-4251-955d-045fb08625d7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.024824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60af9f-8a50-4251-955d-045fb08625d7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.025300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60af9f-8a50-4251-955d-045fb08625d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.032359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:58.035099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 150/297] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:58.047968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cc60af9f-8a50-4251-955d-045fb08625d7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:58.049563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cc60af9f-8a50-4251-955d-045fb08625d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:58.123745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 148/298] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:13:58.162728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc78749-9d0f-4a1b-bdcb-57eb375aebb0 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.163056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc78749-9d0f-4a1b-bdcb-57eb375aebb0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.163284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc78749-9d0f-4a1b-bdcb-57eb375aebb0 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.164615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 151/299] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:58.170271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c1d2c-6592-443f-aaf9-fa390540d4d2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.170526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c1d2c-6592-443f-aaf9-fa390540d4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.170799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c1d2c-6592-443f-aaf9-fa390540d4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.189197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b25c1d2c-6592-443f-aaf9-fa390540d4d2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:58.190634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b25c1d2c-6592-443f-aaf9-fa390540d4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:58.266978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 149/300] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:58.327052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc78749-9d0f-4a1b-bdcb-57eb375aebb0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:58.348961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:58.409695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c1d2c-6592-443f-aaf9-fa390540d4d2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:58.439857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc78749-9d0f-4a1b-bdcb-57eb375aebb0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:58.480757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:58.481842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed7074-5fbc-47fe-b57e-f9b7f4c560a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:58.482611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed7074-5fbc-47fe-b57e-f9b7f4c560a7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.482898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed7074-5fbc-47fe-b57e-f9b7f4c560a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.483150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed7074-5fbc-47fe-b57e-f9b7f4c560a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.495750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed7074-5fbc-47fe-b57e-f9b7f4c560a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3fed7074-5fbc-47fe-b57e-f9b7f4c560a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:58.501486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:58.502386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:13:58.503141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 152/301] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:13:58.508642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed7074-5fbc-47fe-b57e-f9b7f4c560a7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:58.522244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c1d2c-6592-443f-aaf9-fa390540d4d2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:58.557382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e45f46-9394-4aa3-9860-35286c6404f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:58.558048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e45f46-9394-4aa3-9860-35286c6404f3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.558332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e45f46-9394-4aa3-9860-35286c6404f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.558609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e45f46-9394-4aa3-9860-35286c6404f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.569601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e45f46-9394-4aa3-9860-35286c6404f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-86e45f46-9394-4aa3-9860-35286c6404f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:13:58.575070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:13:58.577116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 150/302] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:13:58.577874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e45f46-9394-4aa3-9860-35286c6404f3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:58.606884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:58.633046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed7074-5fbc-47fe-b57e-f9b7f4c560a7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:58.669311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb362dbe-7a19-47b4-8213-734f6aa36f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:58.670504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb362dbe-7a19-47b4-8213-734f6aa36f4e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.671255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb362dbe-7a19-47b4-8213-734f6aa36f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.671584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb362dbe-7a19-47b4-8213-734f6aa36f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.672658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb362dbe-7a19-47b4-8213-734f6aa36f4e None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:58.672973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb362dbe-7a19-47b4-8213-734f6aa36f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:58.674786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb362dbe-7a19-47b4-8213-734f6aa36f4e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:58.684946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 153/303] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:58.690845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb362dbe-7a19-47b4-8213-734f6aa36f4e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:58.697541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e45f46-9394-4aa3-9860-35286c6404f3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:58.721459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:58.735661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b58c8-e7e3-41fd-9353-7111809c1ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:58.736560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b58c8-e7e3-41fd-9353-7111809c1ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.736845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b58c8-e7e3-41fd-9353-7111809c1ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.737127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b58c8-e7e3-41fd-9353-7111809c1ae7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.739240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b58c8-e7e3-41fd-9353-7111809c1ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:58.741227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b58c8-e7e3-41fd-9353-7111809c1ae7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:58.743438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 151/304] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:13:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:58.744004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b58c8-e7e3-41fd-9353-7111809c1ae7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:58.797285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb362dbe-7a19-47b4-8213-734f6aa36f4e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:58.831647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccd1eb-79a1-44ff-a472-e3d99df191f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:58.832247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccd1eb-79a1-44ff-a472-e3d99df191f5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.832525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccd1eb-79a1-44ff-a472-e3d99df191f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.832792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccd1eb-79a1-44ff-a472-e3d99df191f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.833432 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccd1eb-79a1-44ff-a472-e3d99df191f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:58.833738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccd1eb-79a1-44ff-a472-e3d99df191f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:58.835111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccd1eb-79a1-44ff-a472-e3d99df191f5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:58.843560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 154/305] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:13:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:58.844147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccd1eb-79a1-44ff-a472-e3d99df191f5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:58.849511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:58.850545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b58c8-e7e3-41fd-9353-7111809c1ae7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:58.884979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db5a57-b0d2-458c-8ca7-a45814185e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:58.885613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db5a57-b0d2-458c-8ca7-a45814185e93 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.885893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db5a57-b0d2-458c-8ca7-a45814185e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.886197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db5a57-b0d2-458c-8ca7-a45814185e93 None admin] PATH_INFO: `/v3/users/cbbadad7e843449ab937a4b0e5f0a6bc` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.887309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db5a57-b0d2-458c-8ca7-a45814185e93 None admin] RBAC: Authorizing `identity:get_user(user_id=cbbadad7e843449ab937a4b0e5f0a6bc)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:58.889371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db5a57-b0d2-458c-8ca7-a45814185e93 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:58.895625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 152/306] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:58 2022] GET /identity/v3/users/cbbadad7e843449ab937a4b0e5f0a6bc => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:58.896325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db5a57-b0d2-458c-8ca7-a45814185e93 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:58.954875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccd1eb-79a1-44ff-a472-e3d99df191f5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:58.967648 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:58.992400 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b8964-ed87-4e30-b49e-5743dce04d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:58.993017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b8964-ed87-4e30-b49e-5743dce04d05 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:58.993300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b8964-ed87-4e30-b49e-5743dce04d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:58.993577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b8964-ed87-4e30-b49e-5743dce04d05 None admin] PATH_INFO: `/v3/groups` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:58.994233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b8964-ed87-4e30-b49e-5743dce04d05 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:58.995701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b8964-ed87-4e30-b49e-5743dce04d05 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:59.025970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 155/307] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Sep 21 05:13:58 2022] POST /identity/v3/groups => generated 229 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:13:59.025970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b8964-ed87-4e30-b49e-5743dce04d05 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:59.035754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db5a57-b0d2-458c-8ca7-a45814185e93 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:59.084807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed89ef-9332-4c20-9917-7b33694e7ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:59.085464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed89ef-9332-4c20-9917-7b33694e7ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:59.086480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed89ef-9332-4c20-9917-7b33694e7ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:59.086768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed89ef-9332-4c20-9917-7b33694e7ec5 None admin] PATH_INFO: `/v3/users/c8d4f3ee6ab64450b038937358ef0dd0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:59.097039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:59.111187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed89ef-9332-4c20-9917-7b33694e7ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=c8d4f3ee6ab64450b038937358ef0dd0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:59.113010 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed89ef-9332-4c20-9917-7b33694e7ec5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:59.114863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 153/308] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:13:58 2022] GET /identity/v3/users/c8d4f3ee6ab64450b038937358ef0dd0 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:59.121114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed89ef-9332-4c20-9917-7b33694e7ec5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:13:59.149139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b8964-ed87-4e30-b49e-5743dce04d05 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:59.190497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32918796-7bef-4474-bde7-3b1ddfbfa169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:59.191179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32918796-7bef-4474-bde7-3b1ddfbfa169 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:59.191804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32918796-7bef-4474-bde7-3b1ddfbfa169 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:59.191804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32918796-7bef-4474-bde7-3b1ddfbfa169 None admin] PATH_INFO: `/v3/system/users/cbbadad7e843449ab937a4b0e5f0a6bc/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:59.193358 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32918796-7bef-4474-bde7-3b1ddfbfa169 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cbbadad7e843449ab937a4b0e5f0a6bc, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:59.194802 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32918796-7bef-4474-bde7-3b1ddfbfa169 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:59.209283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 156/309] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Sep 21 05:13:59 2022] PUT /identity/v3/system/users/cbbadad7e843449ab937a4b0e5f0a6bc/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:13:59.270912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed89ef-9332-4c20-9917-7b33694e7ec5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:13:59.291495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:13:59.313921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b252e4-7ec2-4149-8f5e-65b49672262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:13:59.314557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b252e4-7ec2-4149-8f5e-65b49672262e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:59.314836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b252e4-7ec2-4149-8f5e-65b49672262e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:59.315109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b252e4-7ec2-4149-8f5e-65b49672262e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:59.315848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b252e4-7ec2-4149-8f5e-65b49672262e None admin] RBAC: Adding query filter params (role.id=439053f3a99d43f5abe2fb7dec39df9d, user.id=c8d4f3ee6ab64450b038937358ef0dd0, scope.system=all) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:13:59.316158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b252e4-7ec2-4149-8f5e-65b49672262e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:13:59.317609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b252e4-7ec2-4149-8f5e-65b49672262e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:13:59.323824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 154/310] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Sep 21 05:13:59 2022] GET /identity/v3/role_assignments?user.id=c8d4f3ee6ab64450b038937358ef0dd0&scope.system=all&role.id=439053f3a99d43f5abe2fb7dec39df9d => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:13:59.559223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419aa81-12e8-4220-bb1c-492ac6a5cf3c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:59.559408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419aa81-12e8-4220-bb1c-492ac6a5cf3c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:59.559692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419aa81-12e8-4220-bb1c-492ac6a5cf3c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:59.561821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 157/311] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:13:59.567312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8ec0a-bacb-407f-b9f3-92ba0cde80b4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:13:59.567606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8ec0a-bacb-407f-b9f3-92ba0cde80b4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:13:59.567889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8ec0a-bacb-407f-b9f3-92ba0cde80b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:13:59.596671 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-dbf8ec0a-bacb-407f-b9f3-92ba0cde80b4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:13:59.598155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dbf8ec0a-bacb-407f-b9f3-92ba0cde80b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:13:59.669562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 155/312] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:13:59.938197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419aa81-12e8-4220-bb1c-492ac6a5cf3c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:00.047944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419aa81-12e8-4220-bb1c-492ac6a5cf3c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:00.075514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:00.084445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a2050-2adf-427a-a1b2-2286fbdbf5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:00.085099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a2050-2adf-427a-a1b2-2286fbdbf5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:00.085379 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a2050-2adf-427a-a1b2-2286fbdbf5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:00.085653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a2050-2adf-427a-a1b2-2286fbdbf5c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:00.096701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a2050-2adf-427a-a1b2-2286fbdbf5c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-294a2050-2adf-427a-a1b2-2286fbdbf5c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:00.101439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:00.102332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:00.103110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 158/313] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:13:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:00.108616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8ec0a-bacb-407f-b9f3-92ba0cde80b4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:00.213904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8ec0a-bacb-407f-b9f3-92ba0cde80b4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:00.226669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:00.251110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3e993-283f-4338-9cd9-474d1c44ae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:00.251724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3e993-283f-4338-9cd9-474d1c44ae36 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:00.252004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3e993-283f-4338-9cd9-474d1c44ae36 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:00.252275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3e993-283f-4338-9cd9-474d1c44ae36 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:00.252932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3e993-283f-4338-9cd9-474d1c44ae36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:00.253238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3e993-283f-4338-9cd9-474d1c44ae36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:00.254644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3e993-283f-4338-9cd9-474d1c44ae36 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:00.263139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 156/314] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:00.269294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a2050-2adf-427a-a1b2-2286fbdbf5c1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:00.377306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a2050-2adf-427a-a1b2-2286fbdbf5c1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:00.414939 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fa48e-128b-45cd-957d-8dc0dce6d2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:00.415567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fa48e-128b-45cd-957d-8dc0dce6d2ad None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:00.415843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fa48e-128b-45cd-957d-8dc0dce6d2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:00.416110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fa48e-128b-45cd-957d-8dc0dce6d2ad None admin] PATH_INFO: `/v3/users/2bee92248caf432b9dec0079a72e029a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:00.417230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fa48e-128b-45cd-957d-8dc0dce6d2ad None admin] RBAC: Authorizing `identity:get_user(user_id=2bee92248caf432b9dec0079a72e029a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:00.418679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fa48e-128b-45cd-957d-8dc0dce6d2ad None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:00.420586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 159/315] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:14:00 2022] GET /identity/v3/users/2bee92248caf432b9dec0079a72e029a => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:00.425754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3e993-283f-4338-9cd9-474d1c44ae36 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:00.529689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3e993-283f-4338-9cd9-474d1c44ae36 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:00.563610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215f109-5cda-44aa-b3a5-1ada484f429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:00.564217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215f109-5cda-44aa-b3a5-1ada484f429b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:00.564491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215f109-5cda-44aa-b3a5-1ada484f429b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:00.564766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215f109-5cda-44aa-b3a5-1ada484f429b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:00.609164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:00.635879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215f109-5cda-44aa-b3a5-1ada484f429b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:00.637328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215f109-5cda-44aa-b3a5-1ada484f429b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:00.739822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 157/316] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:00.745404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fa48e-128b-45cd-957d-8dc0dce6d2ad None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:00.802398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:00.847285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fa48e-128b-45cd-957d-8dc0dce6d2ad None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:00.882374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa991a7-7122-458f-bc78-b472dc3c6a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:00.883415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa991a7-7122-458f-bc78-b472dc3c6a5e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:00.883692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa991a7-7122-458f-bc78-b472dc3c6a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:00.883960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa991a7-7122-458f-bc78-b472dc3c6a5e None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:00.885067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa991a7-7122-458f-bc78-b472dc3c6a5e None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:00.886609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa991a7-7122-458f-bc78-b472dc3c6a5e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:00.888409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 160/317] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:00 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:00.894142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215f109-5cda-44aa-b3a5-1ada484f429b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:00.970092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:01.006551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215f109-5cda-44aa-b3a5-1ada484f429b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:01.041763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b090634-d459-4ed7-8b2a-9cfe5ba9db77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:01.042391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b090634-d459-4ed7-8b2a-9cfe5ba9db77 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:01.042665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b090634-d459-4ed7-8b2a-9cfe5ba9db77 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:01.042935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b090634-d459-4ed7-8b2a-9cfe5ba9db77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:01.043656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b090634-d459-4ed7-8b2a-9cfe5ba9db77 None admin] RBAC: Adding query filter params (role.id=439053f3a99d43f5abe2fb7dec39df9d, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, user.id=2bee92248caf432b9dec0079a72e029a) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:01.043958 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b090634-d459-4ed7-8b2a-9cfe5ba9db77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:01.045945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b090634-d459-4ed7-8b2a-9cfe5ba9db77 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:01.052387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 158/318] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:00 2022] GET /identity/v3/role_assignments?user.id=2bee92248caf432b9dec0079a72e029a&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=439053f3a99d43f5abe2fb7dec39df9d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:01.414426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f7fe-7c95-41a9-9e9b-09c4c9fdb586 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:01.414936 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f7fe-7c95-41a9-9e9b-09c4c9fdb586 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:01.415008 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f7fe-7c95-41a9-9e9b-09c4c9fdb586 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:01.416593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 161/319] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:01.422199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2624d7f-96d4-4a24-a9a5-c900882424f3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:01.422566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2624d7f-96d4-4a24-a9a5-c900882424f3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:01.422751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2624d7f-96d4-4a24-a9a5-c900882424f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:01.441006 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f2624d7f-96d4-4a24-a9a5-c900882424f3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:01.442265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f2624d7f-96d4-4a24-a9a5-c900882424f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:01.495334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269288cc-3126-4894-9ba2-3970cb6b6f4f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:01.495649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269288cc-3126-4894-9ba2-3970cb6b6f4f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:01.495904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269288cc-3126-4894-9ba2-3970cb6b6f4f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:01.497348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 162/320] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:01.502179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643868ba-c6fd-49c4-9388-755275d0f27d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:01.502470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643868ba-c6fd-49c4-9388-755275d0f27d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:01.502775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643868ba-c6fd-49c4-9388-755275d0f27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:01.515079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 159/321] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:01.520899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-643868ba-c6fd-49c4-9388-755275d0f27d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:01.522116 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-643868ba-c6fd-49c4-9388-755275d0f27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:01.568766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b337b1-1b32-47c9-9f4a-b6b62f5e1a5a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:01.569066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b337b1-1b32-47c9-9f4a-b6b62f5e1a5a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:01.569319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b337b1-1b32-47c9-9f4a-b6b62f5e1a5a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:01.570784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 160/322] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:14:01.575518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3f58b-3466-47a6-af1c-e80b8831481f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:01.576554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3f58b-3466-47a6-af1c-e80b8831481f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:01.577047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3f58b-3466-47a6-af1c-e80b8831481f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:01.592810 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 163/323] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:14:01.602496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-63d3f58b-3466-47a6-af1c-e80b8831481f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:01.603753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-63d3f58b-3466-47a6-af1c-e80b8831481f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:01.664149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:01.673712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 161/324] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:01.782489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643868ba-c6fd-49c4-9388-755275d0f27d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:01.838432 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:01.864482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3f58b-3466-47a6-af1c-e80b8831481f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:01.885420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643868ba-c6fd-49c4-9388-755275d0f27d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:01.930311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d1202-ebda-41ff-9078-182986b5eb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:01.931417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d1202-ebda-41ff-9078-182986b5eb79 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:01.932377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d1202-ebda-41ff-9078-182986b5eb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:01.932872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d1202-ebda-41ff-9078-182986b5eb79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:01.934559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d1202-ebda-41ff-9078-182986b5eb79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:01.936039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d1202-ebda-41ff-9078-182986b5eb79 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:01.937580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 164/325] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:01.938350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d1202-ebda-41ff-9078-182986b5eb79 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:01.965523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3f58b-3466-47a6-af1c-e80b8831481f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:01.999288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443c7ad-c06f-4ee6-a7b7-d0e2fb29ecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:02.000003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c7ad-c06f-4ee6-a7b7-d0e2fb29ecf4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:02.000293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c7ad-c06f-4ee6-a7b7-d0e2fb29ecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:02.000571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c7ad-c06f-4ee6-a7b7-d0e2fb29ecf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:02.010552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443c7ad-c06f-4ee6-a7b7-d0e2fb29ecf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-d443c7ad-c06f-4ee6-a7b7-d0e2fb29ecf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:02.015916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:02.016827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 162/326] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:02.017779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443c7ad-c06f-4ee6-a7b7-d0e2fb29ecf4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:02.039099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d1202-ebda-41ff-9078-182986b5eb79 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:02.061558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:02.072537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:02.073774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275897d-bd3c-4339-85b9-d303bb5f6e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:02.074566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275897d-bd3c-4339-85b9-d303bb5f6e42 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:02.074874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275897d-bd3c-4339-85b9-d303bb5f6e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:02.075145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275897d-bd3c-4339-85b9-d303bb5f6e42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:02.085868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275897d-bd3c-4339-85b9-d303bb5f6e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-2275897d-bd3c-4339-85b9-d303bb5f6e42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:02.090619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:02.091454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:02.092236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 165/327] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:02.092873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275897d-bd3c-4339-85b9-d303bb5f6e42 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:02.117573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443c7ad-c06f-4ee6-a7b7-d0e2fb29ecf4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:02.151572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590f09b-8972-4208-8f18-441fde04bb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:02.152192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590f09b-8972-4208-8f18-441fde04bb58 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:02.152460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590f09b-8972-4208-8f18-441fde04bb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:02.152730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590f09b-8972-4208-8f18-441fde04bb58 None admin] PATH_INFO: `/v3/groups` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:02.153344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590f09b-8972-4208-8f18-441fde04bb58 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:02.154949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590f09b-8972-4208-8f18-441fde04bb58 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:02.170522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 163/328] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Sep 21 05:14:02 2022] POST /identity/v3/groups => generated 226 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:14:02.171970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590f09b-8972-4208-8f18-441fde04bb58 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:02.196791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275897d-bd3c-4339-85b9-d303bb5f6e42 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:02.232951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c417de1-83b3-46bf-9e86-42d562181d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:02.233572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c417de1-83b3-46bf-9e86-42d562181d28 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:02.233848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c417de1-83b3-46bf-9e86-42d562181d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:02.234143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c417de1-83b3-46bf-9e86-42d562181d28 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:02.234817 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c417de1-83b3-46bf-9e86-42d562181d28 None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:02.235120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c417de1-83b3-46bf-9e86-42d562181d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:02.236512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c417de1-83b3-46bf-9e86-42d562181d28 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:02.245750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 166/329] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:02.251955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c417de1-83b3-46bf-9e86-42d562181d28 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:02.300994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:02.321415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590f09b-8972-4208-8f18-441fde04bb58 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:02.341159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:02.365176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3e39-6ebc-47c4-aecb-fbe3211c6b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:02.365894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3e39-6ebc-47c4-aecb-fbe3211c6b23 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:02.366127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3e39-6ebc-47c4-aecb-fbe3211c6b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:02.366403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3e39-6ebc-47c4-aecb-fbe3211c6b23 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:02.367105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3e39-6ebc-47c4-aecb-fbe3211c6b23 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:02.367415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3e39-6ebc-47c4-aecb-fbe3211c6b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:02.368113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c417de1-83b3-46bf-9e86-42d562181d28 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:02.369371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3e39-6ebc-47c4-aecb-fbe3211c6b23 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:02.378978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 164/330] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:02.384782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3e39-6ebc-47c4-aecb-fbe3211c6b23 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:02.407369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59693ac0-5c75-42a0-bc46-9528fb982b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:02.408452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59693ac0-5c75-42a0-bc46-9528fb982b42 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:02.408743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59693ac0-5c75-42a0-bc46-9528fb982b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:02.409022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59693ac0-5c75-42a0-bc46-9528fb982b42 None admin] PATH_INFO: `/v3/users/cbbadad7e843449ab937a4b0e5f0a6bc` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:02.410334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59693ac0-5c75-42a0-bc46-9528fb982b42 None admin] RBAC: Authorizing `identity:get_user(user_id=cbbadad7e843449ab937a4b0e5f0a6bc)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:02.411772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59693ac0-5c75-42a0-bc46-9528fb982b42 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:02.413670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 167/331] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:14:02 2022] GET /identity/v3/users/cbbadad7e843449ab937a4b0e5f0a6bc => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:02.422075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59693ac0-5c75-42a0-bc46-9528fb982b42 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:02.507152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3e39-6ebc-47c4-aecb-fbe3211c6b23 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:02.549999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57361c29-4357-4ac3-8651-43fb4f86bbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:02.551018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57361c29-4357-4ac3-8651-43fb4f86bbcd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:02.551378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57361c29-4357-4ac3-8651-43fb4f86bbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:02.551725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57361c29-4357-4ac3-8651-43fb4f86bbcd None admin] PATH_INFO: `/v3/users/c8d4f3ee6ab64450b038937358ef0dd0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:02.553187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59693ac0-5c75-42a0-bc46-9528fb982b42 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:02.554840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57361c29-4357-4ac3-8651-43fb4f86bbcd None admin] RBAC: Authorizing `identity:get_user(user_id=c8d4f3ee6ab64450b038937358ef0dd0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:02.555633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57361c29-4357-4ac3-8651-43fb4f86bbcd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:02.559596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 165/332] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:14:02 2022] GET /identity/v3/users/c8d4f3ee6ab64450b038937358ef0dd0 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:02.566581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57361c29-4357-4ac3-8651-43fb4f86bbcd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:02.569206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:02.596406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7c9c4-e001-4c9b-bc70-7f7ac4fe0041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:02.596727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7c9c4-e001-4c9b-bc70-7f7ac4fe0041 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:02.597115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7c9c4-e001-4c9b-bc70-7f7ac4fe0041 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:02.597390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7c9c4-e001-4c9b-bc70-7f7ac4fe0041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:02.601316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7c9c4-e001-4c9b-bc70-7f7ac4fe0041 None admin] RBAC: Adding query filter params (role.id=d36487377a664531a7f2e06f2f648a6f, user.id=cbbadad7e843449ab937a4b0e5f0a6bc, scope.system=all) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:02.601316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7c9c4-e001-4c9b-bc70-7f7ac4fe0041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:02.601316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7c9c4-e001-4c9b-bc70-7f7ac4fe0041 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:02.617186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 168/333] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Sep 21 05:14:02 2022] GET /identity/v3/role_assignments?user.id=cbbadad7e843449ab937a4b0e5f0a6bc&scope.system=all&role.id=d36487377a664531a7f2e06f2f648a6f => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:02.630191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:02.695169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57361c29-4357-4ac3-8651-43fb4f86bbcd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:02.738890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b33a8-70ec-4494-8f1b-3375eb6b152b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:02.739568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b33a8-70ec-4494-8f1b-3375eb6b152b None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:02.739851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b33a8-70ec-4494-8f1b-3375eb6b152b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:02.740130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b33a8-70ec-4494-8f1b-3375eb6b152b None admin] PATH_INFO: `/v3/system/users/c8d4f3ee6ab64450b038937358ef0dd0/roles/439053f3a99d43f5abe2fb7dec39df9d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:02.742738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b33a8-70ec-4494-8f1b-3375eb6b152b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c8d4f3ee6ab64450b038937358ef0dd0, role_id=439053f3a99d43f5abe2fb7dec39df9d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:02.744178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b33a8-70ec-4494-8f1b-3375eb6b152b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:02.758897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 166/334] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Sep 21 05:14:02 2022] PUT /identity/v3/system/users/c8d4f3ee6ab64450b038937358ef0dd0/roles/439053f3a99d43f5abe2fb7dec39df9d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:14:04.900047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2b2d4-0370-4cfe-805b-e914c2c2b45c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:04.901036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2b2d4-0370-4cfe-805b-e914c2c2b45c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:04.901525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2b2d4-0370-4cfe-805b-e914c2c2b45c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:04.903390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 169/335] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:04.909778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46997fec-a7cf-40d6-8590-48fd8d0a39f0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:04.910706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46997fec-a7cf-40d6-8590-48fd8d0a39f0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:04.911198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46997fec-a7cf-40d6-8590-48fd8d0a39f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:04.932574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-46997fec-a7cf-40d6-8590-48fd8d0a39f0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:04.933810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-46997fec-a7cf-40d6-8590-48fd8d0a39f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:05.027353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 167/336] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:05.177592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c896a-95bc-414c-aa64-a3d76cc725e9 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:05.178215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c896a-95bc-414c-aa64-a3d76cc725e9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:05.178715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c896a-95bc-414c-aa64-a3d76cc725e9 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:05.180245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 170/337] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:05.186247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d0f73-7c67-4098-a8b7-6b7a6598793c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:05.186534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d0f73-7c67-4098-a8b7-6b7a6598793c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:05.186808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d0f73-7c67-4098-a8b7-6b7a6598793c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:05.211032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a70d0f73-7c67-4098-a8b7-6b7a6598793c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:05.212234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a70d0f73-7c67-4098-a8b7-6b7a6598793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:05.271691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:05.312561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 168/338] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:14:05.596546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248d973-f56a-4f3f-8c99-fca109d1b35e None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:05.597460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248d973-f56a-4f3f-8c99-fca109d1b35e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:05.597980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248d973-f56a-4f3f-8c99-fca109d1b35e None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:05.599646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 171/339] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:05 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:05.606376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45f459-055d-4400-ab01-f4a0d9338a01 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:05.607575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45f459-055d-4400-ab01-f4a0d9338a01 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:05.608082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45f459-055d-4400-ab01-f4a0d9338a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:05.631330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3c45f459-055d-4400-ab01-f4a0d9338a01 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:05.633604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248d973-f56a-4f3f-8c99-fca109d1b35e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:05.635112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3c45f459-055d-4400-ab01-f4a0d9338a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:05.752047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 169/340] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:05.774854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0dd63-8b59-4027-bd20-9d13a7bc58e5 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:05.775622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0dd63-8b59-4027-bd20-9d13a7bc58e5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:05.776103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0dd63-8b59-4027-bd20-9d13a7bc58e5 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:05.781099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 170/341] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:14:05.786765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26998edb-c2ee-40e4-802e-d7c9d89b1ffc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:05.786992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26998edb-c2ee-40e4-802e-d7c9d89b1ffc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:05.787380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26998edb-c2ee-40e4-802e-d7c9d89b1ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:05.808464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-26998edb-c2ee-40e4-802e-d7c9d89b1ffc None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:05.808978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248d973-f56a-4f3f-8c99-fca109d1b35e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:05.809812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-26998edb-c2ee-40e4-802e-d7c9d89b1ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:05.879419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f087d-5bfc-468d-94f9-062e1fa30a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:05.880462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f087d-5bfc-468d-94f9-062e1fa30a75 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:05.881025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f087d-5bfc-468d-94f9-062e1fa30a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:05.881476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f087d-5bfc-468d-94f9-062e1fa30a75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:05.885211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 171/342] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:14:05.886245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26998edb-c2ee-40e4-802e-d7c9d89b1ffc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:05.911370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f087d-5bfc-468d-94f9-062e1fa30a75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5c4f087d-5bfc-468d-94f9-062e1fa30a75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:05.921958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:05.922865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:05.923636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:05.926466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 172/343] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:14:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:05.931829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f087d-5bfc-468d-94f9-062e1fa30a75 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:05.946319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:05.995449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:06.052064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26998edb-c2ee-40e4-802e-d7c9d89b1ffc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:06.116429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26224f90-666e-4681-a175-bae48ecff119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:06.117069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26224f90-666e-4681-a175-bae48ecff119 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:06.117354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26224f90-666e-4681-a175-bae48ecff119 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:06.117630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26224f90-666e-4681-a175-bae48ecff119 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:06.129493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f087d-5bfc-468d-94f9-062e1fa30a75 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:06.146841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26224f90-666e-4681-a175-bae48ecff119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-26224f90-666e-4681-a175-bae48ecff119 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:06.152134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:06.153027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:06.153863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:06.156987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 172/344] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:06.162312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26224f90-666e-4681-a175-bae48ecff119 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:06.169911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdce185-2b11-4735-9b87-973a7818296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:06.170543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdce185-2b11-4735-9b87-973a7818296b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:06.170822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdce185-2b11-4735-9b87-973a7818296b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:06.171092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdce185-2b11-4735-9b87-973a7818296b None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:06.171743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdce185-2b11-4735-9b87-973a7818296b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:06.172043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdce185-2b11-4735-9b87-973a7818296b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:06.173463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdce185-2b11-4735-9b87-973a7818296b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:06.184120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 173/345] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:06.189441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdce185-2b11-4735-9b87-973a7818296b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:06.318325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:06.328874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:06.331748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdce185-2b11-4735-9b87-973a7818296b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:06.346139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26224f90-666e-4681-a175-bae48ecff119 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:06.371640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640e5b6-1ba2-4808-8f61-7047d11349f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:06.372271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640e5b6-1ba2-4808-8f61-7047d11349f8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:06.372571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640e5b6-1ba2-4808-8f61-7047d11349f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:06.372924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640e5b6-1ba2-4808-8f61-7047d11349f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:06.455335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82e0c0-2325-49c0-be43-321d30447d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:06.455684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640e5b6-1ba2-4808-8f61-7047d11349f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:06.455871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82e0c0-2325-49c0-be43-321d30447d2d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:06.456161 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82e0c0-2325-49c0-be43-321d30447d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:06.456443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82e0c0-2325-49c0-be43-321d30447d2d None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:06.457120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82e0c0-2325-49c0-be43-321d30447d2d None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:06.457418 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640e5b6-1ba2-4808-8f61-7047d11349f8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:06.457518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82e0c0-2325-49c0-be43-321d30447d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:06.458822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82e0c0-2325-49c0-be43-321d30447d2d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:06.484360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 173/346] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:06.485238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82e0c0-2325-49c0-be43-321d30447d2d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:06.574945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:06.588820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 174/347] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 05:14:06.589716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640e5b6-1ba2-4808-8f61-7047d11349f8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:06.637439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82e0c0-2325-49c0-be43-321d30447d2d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:06.681392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eeef07-5bac-45c9-922d-b586362f9599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:06.682156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eeef07-5bac-45c9-922d-b586362f9599 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:06.682545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eeef07-5bac-45c9-922d-b586362f9599 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:06.682970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eeef07-5bac-45c9-922d-b586362f9599 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:06.694792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eeef07-5bac-45c9-922d-b586362f9599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-66eeef07-5bac-45c9-922d-b586362f9599 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:06.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:06.701351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:06.702647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 174/348] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:06.703761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eeef07-5bac-45c9-922d-b586362f9599 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:06.727461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:06.744898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640e5b6-1ba2-4808-8f61-7047d11349f8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:06.791545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19fbee-607a-48b9-88f9-690438bb0f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:06.792811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19fbee-607a-48b9-88f9-690438bb0f1c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:06.793499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19fbee-607a-48b9-88f9-690438bb0f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:06.794334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19fbee-607a-48b9-88f9-690438bb0f1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:06.810765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19fbee-607a-48b9-88f9-690438bb0f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ea19fbee-607a-48b9-88f9-690438bb0f1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:06.817300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:06.818259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:06.819011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:06.819656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 175/349] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:06.821226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81324f8-5a5d-41bb-b702-c25d9f95bca3 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:06.821518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81324f8-5a5d-41bb-b702-c25d9f95bca3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:06.821796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81324f8-5a5d-41bb-b702-c25d9f95bca3 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:06.823193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 176/350] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:06.826882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81324f8-5a5d-41bb-b702-c25d9f95bca3 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:06.856028 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eeef07-5bac-45c9-922d-b586362f9599 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:06.905542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb970806-8ddd-42fb-b31b-dfd2b4f7f69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:06.906655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb970806-8ddd-42fb-b31b-dfd2b4f7f69e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:06.906938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb970806-8ddd-42fb-b31b-dfd2b4f7f69e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:06.907212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb970806-8ddd-42fb-b31b-dfd2b4f7f69e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:06.912011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:06.985349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81324f8-5a5d-41bb-b702-c25d9f95bca3 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:07.023290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb970806-8ddd-42fb-b31b-dfd2b4f7f69e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:07.024777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb970806-8ddd-42fb-b31b-dfd2b4f7f69e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:07.031668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e228d65-63f0-43e1-a9a0-a61caf2243b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:07.032310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e228d65-63f0-43e1-a9a0-a61caf2243b2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.032579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e228d65-63f0-43e1-a9a0-a61caf2243b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.032847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e228d65-63f0-43e1-a9a0-a61caf2243b2 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.035646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e228d65-63f0-43e1-a9a0-a61caf2243b2 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:07.037084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e228d65-63f0-43e1-a9a0-a61caf2243b2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:07.038953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 177/351] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:06 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:07.039490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e228d65-63f0-43e1-a9a0-a61caf2243b2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:07.052002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:07.163339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 175/352] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:07.165308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb970806-8ddd-42fb-b31b-dfd2b4f7f69e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:07.185293 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e228d65-63f0-43e1-a9a0-a61caf2243b2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:07.222238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:07.238247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4613eb-7f77-4214-a689-97568c86f136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:07.239139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4613eb-7f77-4214-a689-97568c86f136 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.239611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4613eb-7f77-4214-a689-97568c86f136 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.240059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4613eb-7f77-4214-a689-97568c86f136 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.240874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4613eb-7f77-4214-a689-97568c86f136 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:07.241318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4613eb-7f77-4214-a689-97568c86f136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:07.242850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4613eb-7f77-4214-a689-97568c86f136 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:07.255789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 178/353] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:07.257356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8f2dd-8c87-4f20-8fcc-45c226364115 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.257843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8f2dd-8c87-4f20-8fcc-45c226364115 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.258903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8f2dd-8c87-4f20-8fcc-45c226364115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.282749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3ff8f2dd-8c87-4f20-8fcc-45c226364115 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:07.284036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3ff8f2dd-8c87-4f20-8fcc-45c226364115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:07.304661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb970806-8ddd-42fb-b31b-dfd2b4f7f69e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:07.323705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:07.356385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22a7b-3dcb-49c3-aa99-658d96a995f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:07.357051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22a7b-3dcb-49c3-aa99-658d96a995f5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.357329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22a7b-3dcb-49c3-aa99-658d96a995f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.357595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22a7b-3dcb-49c3-aa99-658d96a995f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.358715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf22a7b-3dcb-49c3-aa99-658d96a995f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:07.358715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf22a7b-3dcb-49c3-aa99-658d96a995f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:07.360009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf22a7b-3dcb-49c3-aa99-658d96a995f5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:07.369445 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 176/355] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:07.369626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 179/355] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:07.370368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22a7b-3dcb-49c3-aa99-658d96a995f5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:07.370946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8f2dd-8c87-4f20-8fcc-45c226364115 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:07.488060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8f2dd-8c87-4f20-8fcc-45c226364115 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:07.503581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:07.512181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22a7b-3dcb-49c3-aa99-658d96a995f5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:07.529446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8dd5b-7edb-41c9-83b1-da9e42903bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:07.530611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8dd5b-7edb-41c9-83b1-da9e42903bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.530611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8dd5b-7edb-41c9-83b1-da9e42903bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.530862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8dd5b-7edb-41c9-83b1-da9e42903bb9 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.534050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8dd5b-7edb-41c9-83b1-da9e42903bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:07.535483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8dd5b-7edb-41c9-83b1-da9e42903bb9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:07.537259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 180/356] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:07 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:07.537843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8dd5b-7edb-41c9-83b1-da9e42903bb9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:07.565045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150702da-c312-441e-a3ec-98bd7d306949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:07.565045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150702da-c312-441e-a3ec-98bd7d306949 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.565045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150702da-c312-441e-a3ec-98bd7d306949 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.565045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150702da-c312-441e-a3ec-98bd7d306949 None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.572512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150702da-c312-441e-a3ec-98bd7d306949 None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:07.574615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150702da-c312-441e-a3ec-98bd7d306949 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:07.577734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 177/357] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:07 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:07.578544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150702da-c312-441e-a3ec-98bd7d306949 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:07.631212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:07.674038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8dd5b-7edb-41c9-83b1-da9e42903bb9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:07.704659 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150702da-c312-441e-a3ec-98bd7d306949 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:07.714636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087385bf-26d0-4fb3-bc7a-3071f80ebf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:07.715272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087385bf-26d0-4fb3-bc7a-3071f80ebf91 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.715550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087385bf-26d0-4fb3-bc7a-3071f80ebf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.715820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087385bf-26d0-4fb3-bc7a-3071f80ebf91 None admin] PATH_INFO: `/v3/users/c8d4f3ee6ab64450b038937358ef0dd0` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.717323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087385bf-26d0-4fb3-bc7a-3071f80ebf91 None admin] RBAC: Authorizing `identity:get_user(user_id=c8d4f3ee6ab64450b038937358ef0dd0)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:07.718449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087385bf-26d0-4fb3-bc7a-3071f80ebf91 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:07.720203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 181/358] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:14:07 2022] GET /identity/v3/users/c8d4f3ee6ab64450b038937358ef0dd0 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:07.721233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1a86d-b31b-4e19-b548-b72aeb867a2e None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.721415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1a86d-b31b-4e19-b548-b72aeb867a2e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.721639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1a86d-b31b-4e19-b548-b72aeb867a2e None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.727021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 182/359] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:07.732557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1a86d-b31b-4e19-b548-b72aeb867a2e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:07.743408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4367df9-4508-42a9-9ed4-624d45b25dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:07.743408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4367df9-4508-42a9-9ed4-624d45b25dde None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.743408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4367df9-4508-42a9-9ed4-624d45b25dde None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.743408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4367df9-4508-42a9-9ed4-624d45b25dde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.799594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:07.821185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4367df9-4508-42a9-9ed4-624d45b25dde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:07.822794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4367df9-4508-42a9-9ed4-624d45b25dde None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:07.866789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1a86d-b31b-4e19-b548-b72aeb867a2e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:07.868837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:07.912105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52caa966-6489-4142-b250-0438c0e0e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:07.915197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52caa966-6489-4142-b250-0438c0e0e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:07.915197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52caa966-6489-4142-b250-0438c0e0e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:07.915197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52caa966-6489-4142-b250-0438c0e0e7f8 None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:07.916347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52caa966-6489-4142-b250-0438c0e0e7f8 None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:07.920644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52caa966-6489-4142-b250-0438c0e0e7f8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:07.920644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 183/360] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:07 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:07.920895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52caa966-6489-4142-b250-0438c0e0e7f8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:07.943392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 178/361] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:07.944075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4367df9-4508-42a9-9ed4-624d45b25dde None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:08.033813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52caa966-6489-4142-b250-0438c0e0e7f8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:08.051825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4367df9-4508-42a9-9ed4-624d45b25dde None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:08.054135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:08.074661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cb62c-18ed-4c42-b153-1fc05c3ddc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:08.075089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cb62c-18ed-4c42-b153-1fc05c3ddc7c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.075375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cb62c-18ed-4c42-b153-1fc05c3ddc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.075702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cb62c-18ed-4c42-b153-1fc05c3ddc7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.076928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cb62c-18ed-4c42-b153-1fc05c3ddc7c None admin] RBAC: Adding query filter params (group.id=5be88540f41f47679a98363c19a31fec, role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=ad635390dd3d4dcf897fa77044fd970e) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:08.077237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cb62c-18ed-4c42-b153-1fc05c3ddc7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:08.078692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cb62c-18ed-4c42-b153-1fc05c3ddc7c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:08.084655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 184/362] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:07 2022] GET /identity/v3/role_assignments?group.id=5be88540f41f47679a98363c19a31fec&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:08.090078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cb62c-18ed-4c42-b153-1fc05c3ddc7c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:08.098664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee39e5-41d7-48bb-8db1-aa186d50e358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:08.099665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee39e5-41d7-48bb-8db1-aa186d50e358 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.099949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee39e5-41d7-48bb-8db1-aa186d50e358 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.100500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee39e5-41d7-48bb-8db1-aa186d50e358 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.113200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:08.118405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee39e5-41d7-48bb-8db1-aa186d50e358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-22ee39e5-41d7-48bb-8db1-aa186d50e358 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:08.125609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:08.126882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:08.127853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:08.129441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 179/363] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:14:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:08.131119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f160609-9f7a-499f-a00a-611d55b05cb2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.131119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f160609-9f7a-499f-a00a-611d55b05cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.131440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f160609-9f7a-499f-a00a-611d55b05cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.154646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5f160609-9f7a-499f-a00a-611d55b05cb2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:08.155987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f160609-9f7a-499f-a00a-611d55b05cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:08.232772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cb62c-18ed-4c42-b153-1fc05c3ddc7c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:08.255581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 180/364] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:08.257108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f160609-9f7a-499f-a00a-611d55b05cb2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:08.279933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea4978-048e-44e9-9b87-ff6e48c37e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:08.280897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea4978-048e-44e9-9b87-ff6e48c37e56 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.281255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea4978-048e-44e9-9b87-ff6e48c37e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.281603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea4978-048e-44e9-9b87-ff6e48c37e56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.282905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea4978-048e-44e9-9b87-ff6e48c37e56 None admin] RBAC: Adding query filter params (role.id=439053f3a99d43f5abe2fb7dec39df9d, user.id=c8d4f3ee6ab64450b038937358ef0dd0, scope.system=all) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:08.283294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea4978-048e-44e9-9b87-ff6e48c37e56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:08.284868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea4978-048e-44e9-9b87-ff6e48c37e56 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:08.307374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 185/365] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Sep 21 05:14:08 2022] GET /identity/v3/role_assignments?user.id=c8d4f3ee6ab64450b038937358ef0dd0&scope.system=all&role.id=439053f3a99d43f5abe2fb7dec39df9d => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:08.308320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea4978-048e-44e9-9b87-ff6e48c37e56 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:08.359659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:08.385433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f160609-9f7a-499f-a00a-611d55b05cb2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:08.423286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:08.433726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea4978-048e-44e9-9b87-ff6e48c37e56 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:08.455441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ec2a-50c2-474a-946c-eb8c73687191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:08.456071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ec2a-50c2-474a-946c-eb8c73687191 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.456348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ec2a-50c2-474a-946c-eb8c73687191 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.456685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ec2a-50c2-474a-946c-eb8c73687191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.457521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246ec2a-50c2-474a-946c-eb8c73687191 None admin] RBAC: Adding query filter params (group.id=5be88540f41f47679a98363c19a31fec, role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=ad635390dd3d4dcf897fa77044fd970e) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:08.458068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246ec2a-50c2-474a-946c-eb8c73687191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:08.460184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246ec2a-50c2-474a-946c-eb8c73687191 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:08.475802 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec5f92-2dba-4a26-9487-120dd6faf310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:08.476389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 181/366] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:08 2022] GET /identity/v3/role_assignments?group.id=5be88540f41f47679a98363c19a31fec&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=d36487377a664531a7f2e06f2f648a6f => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:08.476511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5f92-2dba-4a26-9487-120dd6faf310 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.476721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5f92-2dba-4a26-9487-120dd6faf310 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.476998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5f92-2dba-4a26-9487-120dd6faf310 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.477104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ec2a-50c2-474a-946c-eb8c73687191 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:08.478363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec5f92-2dba-4a26-9487-120dd6faf310 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:08.479805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec5f92-2dba-4a26-9487-120dd6faf310 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:08.481712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 186/367] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:08 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:08.489397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec5f92-2dba-4a26-9487-120dd6faf310 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:08.628050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec5f92-2dba-4a26-9487-120dd6faf310 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:08.642374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:08.691821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ec2a-50c2-474a-946c-eb8c73687191 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:08.698225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c539c1-a54d-4237-8b5f-966df2b70402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:08.698995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c539c1-a54d-4237-8b5f-966df2b70402 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.699282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c539c1-a54d-4237-8b5f-966df2b70402 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.699552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c539c1-a54d-4237-8b5f-966df2b70402 None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.700800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c539c1-a54d-4237-8b5f-966df2b70402 None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:08.702341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c539c1-a54d-4237-8b5f-966df2b70402 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:08.705492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 187/368] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:08 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:08.706455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c539c1-a54d-4237-8b5f-966df2b70402 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:08.754284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc20662-3df1-4b9e-bb0a-8a33b4fab600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:08.754906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc20662-3df1-4b9e-bb0a-8a33b4fab600 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.755187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc20662-3df1-4b9e-bb0a-8a33b4fab600 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.755460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc20662-3df1-4b9e-bb0a-8a33b4fab600 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.756133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc20662-3df1-4b9e-bb0a-8a33b4fab600 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:08.756436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc20662-3df1-4b9e-bb0a-8a33b4fab600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:08.757836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc20662-3df1-4b9e-bb0a-8a33b4fab600 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:08.769614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 182/369] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:08.770833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc20662-3df1-4b9e-bb0a-8a33b4fab600 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:08.842977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:08.846334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c539c1-a54d-4237-8b5f-966df2b70402 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:08.888957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b400-7a94-4242-a3d9-f92d551b16fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:08.889593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b400-7a94-4242-a3d9-f92d551b16fd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.889870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b400-7a94-4242-a3d9-f92d551b16fd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.890212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b400-7a94-4242-a3d9-f92d551b16fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.905624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667b400-7a94-4242-a3d9-f92d551b16fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:08.906592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc20662-3df1-4b9e-bb0a-8a33b4fab600 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d667b400-7a94-4242-a3d9-f92d551b16fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:08.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:08.911800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 188/370] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:08.913170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b400-7a94-4242-a3d9-f92d551b16fd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:08.954587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e03b3b-b030-4bd5-b9c4-494edbfb255e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:08.955679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e03b3b-b030-4bd5-b9c4-494edbfb255e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:08.956125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e03b3b-b030-4bd5-b9c4-494edbfb255e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:08.956539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e03b3b-b030-4bd5-b9c4-494edbfb255e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:08.957344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:08.958434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e03b3b-b030-4bd5-b9c4-494edbfb255e None admin] RBAC: Adding query filter params (group.id=5be88540f41f47679a98363c19a31fec, role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:08.958927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e03b3b-b030-4bd5-b9c4-494edbfb255e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:08.961348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e03b3b-b030-4bd5-b9c4-494edbfb255e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:08.970476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 183/371] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:08 2022] GET /identity/v3/role_assignments?group.id=5be88540f41f47679a98363c19a31fec&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=d36487377a664531a7f2e06f2f648a6f => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:08.971574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e03b3b-b030-4bd5-b9c4-494edbfb255e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:09.085322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b400-7a94-4242-a3d9-f92d551b16fd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:09.152765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22d14e-990e-49fb-be51-35be0f484678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:09.153477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22d14e-990e-49fb-be51-35be0f484678 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:09.154508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22d14e-990e-49fb-be51-35be0f484678 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:09.154878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22d14e-990e-49fb-be51-35be0f484678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:09.166624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e03b3b-b030-4bd5-b9c4-494edbfb255e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:09.190369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:09.217714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f806aa-1f74-4b28-9244-0a858d2817cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:09.218751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f806aa-1f74-4b28-9244-0a858d2817cb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:09.219033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f806aa-1f74-4b28-9244-0a858d2817cb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:09.219583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f806aa-1f74-4b28-9244-0a858d2817cb None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:09.220280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f806aa-1f74-4b28-9244-0a858d2817cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:09.220586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f806aa-1f74-4b28-9244-0a858d2817cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:09.222189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f806aa-1f74-4b28-9244-0a858d2817cb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:09.233467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 184/372] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:09.239394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f806aa-1f74-4b28-9244-0a858d2817cb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:09.252527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22d14e-990e-49fb-be51-35be0f484678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:09.254254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22d14e-990e-49fb-be51-35be0f484678 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:09.296536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:09.355530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f806aa-1f74-4b28-9244-0a858d2817cb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:09.378718 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 189/373] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:09.384416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22d14e-990e-49fb-be51-35be0f484678 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:09.402664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b213d-bd95-47df-ad2f-1f8043be72e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:09.403378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b213d-bd95-47df-ad2f-1f8043be72e9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:09.403601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b213d-bd95-47df-ad2f-1f8043be72e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:09.404444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b213d-bd95-47df-ad2f-1f8043be72e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:09.460314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:09.489405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b213d-bd95-47df-ad2f-1f8043be72e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:09.491177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b213d-bd95-47df-ad2f-1f8043be72e9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:09.518786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22d14e-990e-49fb-be51-35be0f484678 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:09.575197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c33f-5157-42ee-9686-26846408c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:09.576748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c33f-5157-42ee-9686-26846408c229 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:09.577178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c33f-5157-42ee-9686-26846408c229 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:09.577595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c33f-5157-42ee-9686-26846408c229 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:09.578977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1c33f-5157-42ee-9686-26846408c229 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:09.580570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1c33f-5157-42ee-9686-26846408c229 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:09.582705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 190/374] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:09 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:09.590924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c33f-5157-42ee-9686-26846408c229 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:09.596733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:09.655291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 185/375] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 05:14:09.660928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b213d-bd95-47df-ad2f-1f8043be72e9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:09.781406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:09.799329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c33f-5157-42ee-9686-26846408c229 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:09.804942 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b213d-bd95-47df-ad2f-1f8043be72e9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:09.840904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309151f-6b0c-4db0-9fdc-a05a6fef48e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:09.841557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309151f-6b0c-4db0-9fdc-a05a6fef48e7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:09.841831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309151f-6b0c-4db0-9fdc-a05a6fef48e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:09.842131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309151f-6b0c-4db0-9fdc-a05a6fef48e7 None admin] PATH_INFO: `/v3/projects/01b2fa2971c94d418f0122cbda3c72f9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:09.851643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309151f-6b0c-4db0-9fdc-a05a6fef48e7 None admin] RBAC: Authorizing `identity:get_project(project_id=01b2fa2971c94d418f0122cbda3c72f9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:09.853077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309151f-6b0c-4db0-9fdc-a05a6fef48e7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:09.858316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 186/376] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:09 2022] GET /identity/v3/projects/01b2fa2971c94d418f0122cbda3c72f9 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:09.866799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309151f-6b0c-4db0-9fdc-a05a6fef48e7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:09.872483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bab95-afe7-4bda-b7e4-da4ffc0b4bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:09.873625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bab95-afe7-4bda-b7e4-da4ffc0b4bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:09.873901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bab95-afe7-4bda-b7e4-da4ffc0b4bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:09.874196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bab95-afe7-4bda-b7e4-da4ffc0b4bb9 None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:09.880417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bab95-afe7-4bda-b7e4-da4ffc0b4bb9 None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:09.881850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bab95-afe7-4bda-b7e4-da4ffc0b4bb9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:09.883608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 191/377] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:09 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:09.889167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bab95-afe7-4bda-b7e4-da4ffc0b4bb9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:09.986363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:10.049151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bab95-afe7-4bda-b7e4-da4ffc0b4bb9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:10.095460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309151f-6b0c-4db0-9fdc-a05a6fef48e7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:10.119478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca4d1-02c7-4684-b405-a57a2a0f1cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:10.120421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca4d1-02c7-4684-b405-a57a2a0f1cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:10.121096 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca4d1-02c7-4684-b405-a57a2a0f1cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:10.122355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca4d1-02c7-4684-b405-a57a2a0f1cf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:10.124068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eca4d1-02c7-4684-b405-a57a2a0f1cf9 None admin] RBAC: Adding query filter params (group.id=5be88540f41f47679a98363c19a31fec, role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:10.124595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eca4d1-02c7-4684-b405-a57a2a0f1cf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:10.128784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eca4d1-02c7-4684-b405-a57a2a0f1cf9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:10.140139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 192/378] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:09 2022] GET /identity/v3/role_assignments?group.id=5be88540f41f47679a98363c19a31fec&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:10.172003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bb46a-fc7f-4837-9bca-e3392bc58145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:10.173377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bb46a-fc7f-4837-9bca-e3392bc58145 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:10.173872 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bb46a-fc7f-4837-9bca-e3392bc58145 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:10.174378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bb46a-fc7f-4837-9bca-e3392bc58145 None admin] PATH_INFO: `/v3/groups/a3124b7e11ef4cc2b9c7f0b2bb19d19b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:10.183472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bb46a-fc7f-4837-9bca-e3392bc58145 None admin] RBAC: Authorizing `identity:get_group(group_id=a3124b7e11ef4cc2b9c7f0b2bb19d19b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:10.185879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bb46a-fc7f-4837-9bca-e3392bc58145 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:10.188161 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 187/379] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:09 2022] GET /identity/v3/groups/a3124b7e11ef4cc2b9c7f0b2bb19d19b => generated 229 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:10.193842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca4d1-02c7-4684-b405-a57a2a0f1cf9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:10.308430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca4d1-02c7-4684-b405-a57a2a0f1cf9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:10.347908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3385-97c3-458e-8737-a38249b71029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:10.348924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3385-97c3-458e-8737-a38249b71029 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:10.349413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3385-97c3-458e-8737-a38249b71029 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:10.349896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3385-97c3-458e-8737-a38249b71029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:10.350859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd3385-97c3-458e-8737-a38249b71029 None admin] RBAC: Adding query filter params (group.id=a3124b7e11ef4cc2b9c7f0b2bb19d19b, role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=01b2fa2971c94d418f0122cbda3c72f9) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:10.351410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd3385-97c3-458e-8737-a38249b71029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:10.353128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd3385-97c3-458e-8737-a38249b71029 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:10.355338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:10.360771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 193/380] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:10 2022] GET /identity/v3/role_assignments?group.id=a3124b7e11ef4cc2b9c7f0b2bb19d19b&scope.project.id=01b2fa2971c94d418f0122cbda3c72f9&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:10.784988 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8450124-abe5-47d3-b8b1-dd85b61276e2 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:10.785270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8450124-abe5-47d3-b8b1-dd85b61276e2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:10.785792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8450124-abe5-47d3-b8b1-dd85b61276e2 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:10.787592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 188/381] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:10.792822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbac614-c4f3-465b-aec5-aae4b2481800 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:10.793324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbac614-c4f3-465b-aec5-aae4b2481800 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:10.793841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbac614-c4f3-465b-aec5-aae4b2481800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:10.816483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8bbac614-c4f3-465b-aec5-aae4b2481800 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:10.818007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8bbac614-c4f3-465b-aec5-aae4b2481800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:10.907834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 194/382] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:11.161698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053c3af-90cd-4e53-afa4-dd1eb8aa377a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:11.162389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053c3af-90cd-4e53-afa4-dd1eb8aa377a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:11.162890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053c3af-90cd-4e53-afa4-dd1eb8aa377a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:11.164669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 189/383] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:11.170187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67507278-c96c-47ee-b063-f8e9ad93d7cf None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:11.171051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67507278-c96c-47ee-b063-f8e9ad93d7cf None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:11.171529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67507278-c96c-47ee-b063-f8e9ad93d7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:11.190670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-67507278-c96c-47ee-b063-f8e9ad93d7cf None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:11.192182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-67507278-c96c-47ee-b063-f8e9ad93d7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:11.280234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 195/384] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:11.301397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053c3af-90cd-4e53-afa4-dd1eb8aa377a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:11.328833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:11.458586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053c3af-90cd-4e53-afa4-dd1eb8aa377a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:11.506766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dc4f1-2f44-4dfd-8379-64bc49598df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:11.507400 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dc4f1-2f44-4dfd-8379-64bc49598df8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:11.507680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dc4f1-2f44-4dfd-8379-64bc49598df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:11.507952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dc4f1-2f44-4dfd-8379-64bc49598df8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:11.521775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dc4f1-2f44-4dfd-8379-64bc49598df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-917dc4f1-2f44-4dfd-8379-64bc49598df8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:11.526710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:11.527584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:11.528364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 190/385] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:14:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:11.534362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67507278-c96c-47ee-b063-f8e9ad93d7cf None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:11.604910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:11.651005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67507278-c96c-47ee-b063-f8e9ad93d7cf None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:11.689441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67d3fb-86d1-462c-8950-500bab8248d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:11.690075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67d3fb-86d1-462c-8950-500bab8248d3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:11.690524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67d3fb-86d1-462c-8950-500bab8248d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:11.690801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67d3fb-86d1-462c-8950-500bab8248d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:11.691455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67d3fb-86d1-462c-8950-500bab8248d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:11.691758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67d3fb-86d1-462c-8950-500bab8248d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:11.693104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67d3fb-86d1-462c-8950-500bab8248d3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:11.705947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 196/386] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:11.711513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dc4f1-2f44-4dfd-8379-64bc49598df8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:11.800678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67d3fb-86d1-462c-8950-500bab8248d3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:11.820625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:11.881730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dc4f1-2f44-4dfd-8379-64bc49598df8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:11.936251 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67d3fb-86d1-462c-8950-500bab8248d3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:11.938229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36e8d9-aaab-429c-9b35-23db2b38dfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:11.939048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36e8d9-aaab-429c-9b35-23db2b38dfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:11.939323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36e8d9-aaab-429c-9b35-23db2b38dfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:11.939585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36e8d9-aaab-429c-9b35-23db2b38dfb4 None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:11.940859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36e8d9-aaab-429c-9b35-23db2b38dfb4 None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:11.942528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36e8d9-aaab-429c-9b35-23db2b38dfb4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:11.948922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 191/387] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:11 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:11.950493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82824d-2984-4edf-abb5-d9163e88b4f4 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:11.951482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82824d-2984-4edf-abb5-d9163e88b4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:11.951823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82824d-2984-4edf-abb5-d9163e88b4f4 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:11.953118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 192/388] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:11.954335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82824d-2984-4edf-abb5-d9163e88b4f4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:11.962235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:11.980050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1bf27-29dc-4875-a87a-1f8eb9ce2bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:11.980982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1bf27-29dc-4875-a87a-1f8eb9ce2bca None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:11.981454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1bf27-29dc-4875-a87a-1f8eb9ce2bca None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:11.982035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1bf27-29dc-4875-a87a-1f8eb9ce2bca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.000540 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1bf27-29dc-4875-a87a-1f8eb9ce2bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4de1bf27-29dc-4875-a87a-1f8eb9ce2bca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:12.009403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:12.010369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:12.011163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:12.014718 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 197/389] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:12.016547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c58e68-958b-40c5-a13b-db1280216d1b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:12.017024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c58e68-958b-40c5-a13b-db1280216d1b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:12.017768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c58e68-958b-40c5-a13b-db1280216d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.039859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a0c58e68-958b-40c5-a13b-db1280216d1b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:12.041434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a0c58e68-958b-40c5-a13b-db1280216d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:12.082277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82824d-2984-4edf-abb5-d9163e88b4f4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:12.119715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce1600-d008-4387-9837-618802432daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:12.120658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce1600-d008-4387-9837-618802432daf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:12.121118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce1600-d008-4387-9837-618802432daf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:12.121572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce1600-d008-4387-9837-618802432daf None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.123571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce1600-d008-4387-9837-618802432daf None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:12.125419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce1600-d008-4387-9837-618802432daf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:12.127639 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 193/390] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:11 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:12.128621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce1600-d008-4387-9837-618802432daf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:12.137989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 198/391] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:12.137989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c58e68-958b-40c5-a13b-db1280216d1b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:12.148990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:12.259597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c58e68-958b-40c5-a13b-db1280216d1b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:12.276587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:12.307285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815c389-aec2-4861-84ed-6816133c0728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:12.307959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815c389-aec2-4861-84ed-6816133c0728 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:12.308238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815c389-aec2-4861-84ed-6816133c0728 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:12.308564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815c389-aec2-4861-84ed-6816133c0728 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e/groups/5be88540f41f47679a98363c19a31fec/roles/9578cd8bc0a8498f99757fd0111e48e8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.312595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815c389-aec2-4861-84ed-6816133c0728 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad635390dd3d4dcf897fa77044fd970e, group_id=5be88540f41f47679a98363c19a31fec, role_id=9578cd8bc0a8498f99757fd0111e48e8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:12.314537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815c389-aec2-4861-84ed-6816133c0728 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:12.323220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce1600-d008-4387-9837-618802432daf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:12.345081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 199/392] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Sep 21 05:14:12 2022] PUT /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e/groups/5be88540f41f47679a98363c19a31fec/roles/9578cd8bc0a8498f99757fd0111e48e8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:14:12.371963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fd5d9-ad45-496a-a3db-8c3a71b8b079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:12.373608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fd5d9-ad45-496a-a3db-8c3a71b8b079 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:12.374194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fd5d9-ad45-496a-a3db-8c3a71b8b079 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:12.374771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fd5d9-ad45-496a-a3db-8c3a71b8b079 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.375710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fd5d9-ad45-496a-a3db-8c3a71b8b079 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:12.376358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fd5d9-ad45-496a-a3db-8c3a71b8b079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:12.378181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fd5d9-ad45-496a-a3db-8c3a71b8b079 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:12.388742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 194/393] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:12.395336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815c389-aec2-4861-84ed-6816133c0728 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:12.548586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815c389-aec2-4861-84ed-6816133c0728 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:12.584083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b5900-b146-48d5-a8d8-6226a58f8edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:12.584740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b5900-b146-48d5-a8d8-6226a58f8edb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:12.585018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b5900-b146-48d5-a8d8-6226a58f8edb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:12.585292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b5900-b146-48d5-a8d8-6226a58f8edb None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.586543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b5900-b146-48d5-a8d8-6226a58f8edb None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:12.587995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b5900-b146-48d5-a8d8-6226a58f8edb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:12.589994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 200/394] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:12 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:12.601964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fd5d9-ad45-496a-a3db-8c3a71b8b079 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:12.636883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b5900-b146-48d5-a8d8-6226a58f8edb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:12.683764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:12.700662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:12.741263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fd5d9-ad45-496a-a3db-8c3a71b8b079 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:12.767882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b5900-b146-48d5-a8d8-6226a58f8edb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:12.782548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd340ec-b8fe-4145-895c-aaea13bfed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:12.783615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd340ec-b8fe-4145-895c-aaea13bfed86 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:12.784218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd340ec-b8fe-4145-895c-aaea13bfed86 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:12.784688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd340ec-b8fe-4145-895c-aaea13bfed86 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.786118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd340ec-b8fe-4145-895c-aaea13bfed86 None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:12.787974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd340ec-b8fe-4145-895c-aaea13bfed86 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:12.790113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 195/395] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:12 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:12.797276 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd340ec-b8fe-4145-895c-aaea13bfed86 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:12.828308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00835a4c-8d1a-4abd-9783-bd7bc7c93d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:12.828931 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00835a4c-8d1a-4abd-9783-bd7bc7c93d77 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:12.829214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00835a4c-8d1a-4abd-9783-bd7bc7c93d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:12.829483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00835a4c-8d1a-4abd-9783-bd7bc7c93d77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.854141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00835a4c-8d1a-4abd-9783-bd7bc7c93d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-00835a4c-8d1a-4abd-9783-bd7bc7c93d77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:12.860343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:12.861369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:12.862183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 201/396] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:12.863758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bd6fe-a050-49c1-9deb-2562f6a22085 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:12.863758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bd6fe-a050-49c1-9deb-2562f6a22085 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:12.863925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bd6fe-a050-49c1-9deb-2562f6a22085 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.865197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 202/397] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:12.870968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bd6fe-a050-49c1-9deb-2562f6a22085 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:12.953627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd340ec-b8fe-4145-895c-aaea13bfed86 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:12.990267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7344ca2-070c-42f2-8069-9de81ed074f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:12.991036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7344ca2-070c-42f2-8069-9de81ed074f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:12.991326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7344ca2-070c-42f2-8069-9de81ed074f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:12.991606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7344ca2-070c-42f2-8069-9de81ed074f5 None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e/groups/5be88540f41f47679a98363c19a31fec/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:12.993568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7344ca2-070c-42f2-8069-9de81ed074f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad635390dd3d4dcf897fa77044fd970e, group_id=5be88540f41f47679a98363c19a31fec, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:12.994068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:12.995823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7344ca2-070c-42f2-8069-9de81ed074f5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:12.999347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bd6fe-a050-49c1-9deb-2562f6a22085 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:13.003763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:13.013972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 196/398] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Sep 21 05:14:12 2022] PUT /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e/groups/5be88540f41f47679a98363c19a31fec/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:14:13.013972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857a9e3-c361-4dc7-b2d0-377ed1b430b2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:13.013972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857a9e3-c361-4dc7-b2d0-377ed1b430b2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:13.013972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857a9e3-c361-4dc7-b2d0-377ed1b430b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:13.033892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8857a9e3-c361-4dc7-b2d0-377ed1b430b2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:13.035214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8857a9e3-c361-4dc7-b2d0-377ed1b430b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:13.049615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cb62a-b83d-4da5-b505-8e7a37ea895a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:13.050287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cb62a-b83d-4da5-b505-8e7a37ea895a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:13.050577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cb62a-b83d-4da5-b505-8e7a37ea895a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:13.050848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cb62a-b83d-4da5-b505-8e7a37ea895a None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:13.051519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cb62a-b83d-4da5-b505-8e7a37ea895a None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:13.051822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cb62a-b83d-4da5-b505-8e7a37ea895a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:13.053214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cb62a-b83d-4da5-b505-8e7a37ea895a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:13.063852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 203/399] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:13.066201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccd1df-2d85-4e1b-a043-240a2a889986 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:13.066485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccd1df-2d85-4e1b-a043-240a2a889986 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:13.066755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccd1df-2d85-4e1b-a043-240a2a889986 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:13.068672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 204/400] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:13.069993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfccd1df-2d85-4e1b-a043-240a2a889986 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:13.134412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 197/401] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:13.136095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517ba2b-6ce3-4056-a604-3d4d5912a7f6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:13.136561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517ba2b-6ce3-4056-a604-3d4d5912a7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:13.137082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517ba2b-6ce3-4056-a604-3d4d5912a7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:13.174184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6517ba2b-6ce3-4056-a604-3d4d5912a7f6 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:13.175684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6517ba2b-6ce3-4056-a604-3d4d5912a7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:13.206535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfccd1df-2d85-4e1b-a043-240a2a889986 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:13.250926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5590b-91d4-48ea-9ee3-90dcf65bec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:13.251561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5590b-91d4-48ea-9ee3-90dcf65bec69 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:13.251881 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5590b-91d4-48ea-9ee3-90dcf65bec69 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:13.252153 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5590b-91d4-48ea-9ee3-90dcf65bec69 None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:13.253854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5590b-91d4-48ea-9ee3-90dcf65bec69 None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:13.255870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5590b-91d4-48ea-9ee3-90dcf65bec69 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:13.258838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 205/402] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:13 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:13.264552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5590b-91d4-48ea-9ee3-90dcf65bec69 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:13.269709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 198/403] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:13.301904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:13.402551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5590b-91d4-48ea-9ee3-90dcf65bec69 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:13.448043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517ba2b-6ce3-4056-a604-3d4d5912a7f6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:13.455326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38dcac-8e84-486a-8f41-a7fec7bdfd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:13.455980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38dcac-8e84-486a-8f41-a7fec7bdfd62 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:13.456265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38dcac-8e84-486a-8f41-a7fec7bdfd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:13.456542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38dcac-8e84-486a-8f41-a7fec7bdfd62 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:13.457688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38dcac-8e84-486a-8f41-a7fec7bdfd62 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:13.459190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38dcac-8e84-486a-8f41-a7fec7bdfd62 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:13.461171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 206/404] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:13 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:13.467171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38dcac-8e84-486a-8f41-a7fec7bdfd62 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:13.550721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:13.619677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517ba2b-6ce3-4056-a604-3d4d5912a7f6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:13.642930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:13.650801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38dcac-8e84-486a-8f41-a7fec7bdfd62 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:13.671034 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbb9bd-acfd-4de8-bdc1-a8bf6acb6208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:13.671599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbb9bd-acfd-4de8-bdc1-a8bf6acb6208 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:13.671880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbb9bd-acfd-4de8-bdc1-a8bf6acb6208 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:13.672153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbb9bd-acfd-4de8-bdc1-a8bf6acb6208 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:13.688309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbb9bd-acfd-4de8-bdc1-a8bf6acb6208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-11cbb9bd-acfd-4de8-bdc1-a8bf6acb6208 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:13.694545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:13.695433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:13.696200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:13.700447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 199/405] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:13.701269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbb9bd-acfd-4de8-bdc1-a8bf6acb6208 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:13.703374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44416659-12d0-4fcb-bbd6-e8bc8462e56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:13.704279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44416659-12d0-4fcb-bbd6-e8bc8462e56f None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:13.704739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44416659-12d0-4fcb-bbd6-e8bc8462e56f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:13.705216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44416659-12d0-4fcb-bbd6-e8bc8462e56f None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/groups/5be88540f41f47679a98363c19a31fec/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:13.707575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44416659-12d0-4fcb-bbd6-e8bc8462e56f None admin] RBAC: Authorizing `identity:create_grant(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, group_id=5be88540f41f47679a98363c19a31fec, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:13.709199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44416659-12d0-4fcb-bbd6-e8bc8462e56f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:13.726350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 207/406] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Sep 21 05:14:13 2022] PUT /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/groups/5be88540f41f47679a98363c19a31fec/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:14:13.727588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44416659-12d0-4fcb-bbd6-e8bc8462e56f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:13.915629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbb9bd-acfd-4de8-bdc1-a8bf6acb6208 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:13.943938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:13.954343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44416659-12d0-4fcb-bbd6-e8bc8462e56f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:13.974372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf22f1-67d4-4568-b858-20f357f5f784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:13.975301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf22f1-67d4-4568-b858-20f357f5f784 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:13.975583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf22f1-67d4-4568-b858-20f357f5f784 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:13.975857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf22f1-67d4-4568-b858-20f357f5f784 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:13.992970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf22f1-67d4-4568-b858-20f357f5f784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:13.998008 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7194c-25ca-43fe-b02c-397d35604aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5ccf22f1-67d4-4568-b858-20f357f5f784 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:13.998402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:13.999297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:14.000068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 200/407] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:14:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:14.002913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7194c-25ca-43fe-b02c-397d35604aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:14.003202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7194c-25ca-43fe-b02c-397d35604aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:14.003477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7194c-25ca-43fe-b02c-397d35604aa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:14.004368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7194c-25ca-43fe-b02c-397d35604aa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:14.004548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7194c-25ca-43fe-b02c-397d35604aa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:14.005873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7194c-25ca-43fe-b02c-397d35604aa0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:14.014069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf22f1-67d4-4568-b858-20f357f5f784 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:14.029242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 208/408] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:14.031904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7194c-25ca-43fe-b02c-397d35604aa0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:14.072134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:14.220392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf22f1-67d4-4568-b858-20f357f5f784 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:14.236788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7194c-25ca-43fe-b02c-397d35604aa0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:14.284956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02674d75-9c1a-4112-8043-65bf041f6b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:14.285577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674d75-9c1a-4112-8043-65bf041f6b72 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:14.285853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674d75-9c1a-4112-8043-65bf041f6b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:14.286151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674d75-9c1a-4112-8043-65bf041f6b72 None admin] PATH_INFO: `/v3/groups/a3124b7e11ef4cc2b9c7f0b2bb19d19b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:14.290175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02674d75-9c1a-4112-8043-65bf041f6b72 None admin] RBAC: Authorizing `identity:get_group(group_id=a3124b7e11ef4cc2b9c7f0b2bb19d19b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:14.291607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02674d75-9c1a-4112-8043-65bf041f6b72 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:14.295380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 209/409] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:14 2022] GET /identity/v3/groups/a3124b7e11ef4cc2b9c7f0b2bb19d19b => generated 229 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:14.297765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd407b2-4818-4a63-ba4c-02afc27366fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:14.298408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd407b2-4818-4a63-ba4c-02afc27366fa None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:14.298692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd407b2-4818-4a63-ba4c-02afc27366fa None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:14.298966 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd407b2-4818-4a63-ba4c-02afc27366fa None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:14.299631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd407b2-4818-4a63-ba4c-02afc27366fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:14.299933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd407b2-4818-4a63-ba4c-02afc27366fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:14.301310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd407b2-4818-4a63-ba4c-02afc27366fa None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:14.303427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02674d75-9c1a-4112-8043-65bf041f6b72 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:14.325180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 201/410] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:14.330990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd407b2-4818-4a63-ba4c-02afc27366fa None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:14.411558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:14.435944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02674d75-9c1a-4112-8043-65bf041f6b72 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:14.436370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:14.492773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fca684-9642-45bc-a0c4-b6b8a291e22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:14.493324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fca684-9642-45bc-a0c4-b6b8a291e22e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:14.493657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fca684-9642-45bc-a0c4-b6b8a291e22e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:14.493956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fca684-9642-45bc-a0c4-b6b8a291e22e None admin] PATH_INFO: `/v3/projects/01b2fa2971c94d418f0122cbda3c72f9` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:14.495087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fca684-9642-45bc-a0c4-b6b8a291e22e None admin] RBAC: Authorizing `identity:get_project(project_id=01b2fa2971c94d418f0122cbda3c72f9)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:14.496645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fca684-9642-45bc-a0c4-b6b8a291e22e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:14.498876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 210/411] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:14 2022] GET /identity/v3/projects/01b2fa2971c94d418f0122cbda3c72f9 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:14.504385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fca684-9642-45bc-a0c4-b6b8a291e22e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:14.576364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd407b2-4818-4a63-ba4c-02afc27366fa None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:14.648452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fca684-9642-45bc-a0c4-b6b8a291e22e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:14.659800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3074f-6e84-4bc6-819c-d9aabb00690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:14.660441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3074f-6e84-4bc6-819c-d9aabb00690e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:14.660717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3074f-6e84-4bc6-819c-d9aabb00690e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:14.660986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3074f-6e84-4bc6-819c-d9aabb00690e None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:14.662934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3074f-6e84-4bc6-819c-d9aabb00690e None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:14.664377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3074f-6e84-4bc6-819c-d9aabb00690e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:14.668891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 202/412] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:14 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:14.671512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3074f-6e84-4bc6-819c-d9aabb00690e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:14.722871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed261d-af12-4014-bcf3-b116c1c08149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:14.723567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed261d-af12-4014-bcf3-b116c1c08149 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:14.723849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed261d-af12-4014-bcf3-b116c1c08149 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:14.724126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed261d-af12-4014-bcf3-b116c1c08149 None admin] PATH_INFO: `/v3/projects/01b2fa2971c94d418f0122cbda3c72f9/groups/a3124b7e11ef4cc2b9c7f0b2bb19d19b/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:14.727822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed261d-af12-4014-bcf3-b116c1c08149 None admin] RBAC: Authorizing `identity:create_grant(project_id=01b2fa2971c94d418f0122cbda3c72f9, group_id=a3124b7e11ef4cc2b9c7f0b2bb19d19b, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:14.729464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed261d-af12-4014-bcf3-b116c1c08149 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:14.762484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 211/413] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Sep 21 05:14:14 2022] PUT /identity/v3/projects/01b2fa2971c94d418f0122cbda3c72f9/groups/a3124b7e11ef4cc2b9c7f0b2bb19d19b/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:14:14.838030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3067db4-3084-4522-9c88-d10d4932c58f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:14.838357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3067db4-3084-4522-9c88-d10d4932c58f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:14.838595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3067db4-3084-4522-9c88-d10d4932c58f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:14.839945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 212/414] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:14:14.844503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3074f-6e84-4bc6-819c-d9aabb00690e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:14.845312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880390-13b1-46b1-b04c-47a1baf8c089 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:14.845598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880390-13b1-46b1-b04c-47a1baf8c089 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:14.845872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880390-13b1-46b1-b04c-47a1baf8c089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:14.860339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:14.878377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-29880390-13b1-46b1-b04c-47a1baf8c089 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:14.879660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-29880390-13b1-46b1-b04c-47a1baf8c089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:14.894307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc56aef-5562-450d-bfac-d0c3e944c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:14.894970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc56aef-5562-450d-bfac-d0c3e944c900 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:14.895245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc56aef-5562-450d-bfac-d0c3e944c900 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:14.895618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc56aef-5562-450d-bfac-d0c3e944c900 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:14.896902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc56aef-5562-450d-bfac-d0c3e944c900 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:14.898549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc56aef-5562-450d-bfac-d0c3e944c900 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:14.904186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 203/415] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:14 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:14.904521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:14.907446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc56aef-5562-450d-bfac-d0c3e944c900 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:15.025848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 213/416] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:14:15.057849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc56aef-5562-450d-bfac-d0c3e944c900 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:15.093009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b39b54-d1ac-48dc-9b37-7956104847a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:15.093712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b39b54-d1ac-48dc-9b37-7956104847a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:15.094013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b39b54-d1ac-48dc-9b37-7956104847a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:15.094283 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b39b54-d1ac-48dc-9b37-7956104847a4 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/groups/5be88540f41f47679a98363c19a31fec/roles/9578cd8bc0a8498f99757fd0111e48e8` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:15.096508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b39b54-d1ac-48dc-9b37-7956104847a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2, group_id=5be88540f41f47679a98363c19a31fec, role_id=9578cd8bc0a8498f99757fd0111e48e8)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:15.097957 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b39b54-d1ac-48dc-9b37-7956104847a4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:15.122146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 204/417] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Sep 21 05:14:14 2022] PUT /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2/groups/5be88540f41f47679a98363c19a31fec/roles/9578cd8bc0a8498f99757fd0111e48e8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:14:15.331745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880390-13b1-46b1-b04c-47a1baf8c089 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:15.459691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880390-13b1-46b1-b04c-47a1baf8c089 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:15.516256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c94b5a-6667-417f-bead-0997d98aee90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:15.516917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c94b5a-6667-417f-bead-0997d98aee90 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:15.517194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c94b5a-6667-417f-bead-0997d98aee90 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:15.517460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c94b5a-6667-417f-bead-0997d98aee90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:15.532473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c94b5a-6667-417f-bead-0997d98aee90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-50c94b5a-6667-417f-bead-0997d98aee90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:15.537326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:15.538253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:15.539038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 214/418] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:14:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:15.544822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b39b54-d1ac-48dc-9b37-7956104847a4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:15.598033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:15.610070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de10d7e-8eae-4ac4-8db1-9473259ee2c3 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:15.610356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de10d7e-8eae-4ac4-8db1-9473259ee2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:15.610633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de10d7e-8eae-4ac4-8db1-9473259ee2c3 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:15.612361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 215/419] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:14:15.618010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853be5cc-9184-4853-8cb7-065748333faf None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:15.618303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853be5cc-9184-4853-8cb7-065748333faf None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:15.618581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853be5cc-9184-4853-8cb7-065748333faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:15.626617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:15.638739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-853be5cc-9184-4853-8cb7-065748333faf None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:15.639945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-853be5cc-9184-4853-8cb7-065748333faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:15.712367 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 216/420] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:15.722685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b39b54-d1ac-48dc-9b37-7956104847a4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:15.783623 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581c94b-8c69-48cc-a398-aa023b793fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:15.784278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581c94b-8c69-48cc-a398-aa023b793fcc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:15.784564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581c94b-8c69-48cc-a398-aa023b793fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:15.784840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581c94b-8c69-48cc-a398-aa023b793fcc None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:15.785500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581c94b-8c69-48cc-a398-aa023b793fcc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:15.785801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581c94b-8c69-48cc-a398-aa023b793fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:15.787272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581c94b-8c69-48cc-a398-aa023b793fcc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:15.797776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 205/421] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:15.803817 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853be5cc-9184-4853-8cb7-065748333faf None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:15.925925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853be5cc-9184-4853-8cb7-065748333faf None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:15.960744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b9441-ddec-4f98-98eb-34d421ff5224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:15.961410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b9441-ddec-4f98-98eb-34d421ff5224 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:15.961691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b9441-ddec-4f98-98eb-34d421ff5224 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:15.961977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b9441-ddec-4f98-98eb-34d421ff5224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:15.964755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:16.045305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b9441-ddec-4f98-98eb-34d421ff5224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:16.046774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b9441-ddec-4f98-98eb-34d421ff5224 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:16.157181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581c94b-8c69-48cc-a398-aa023b793fcc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:16.180385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 217/422] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:16.181555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b9441-ddec-4f98-98eb-34d421ff5224 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:16.236618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:16.353925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:16.361511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b9441-ddec-4f98-98eb-34d421ff5224 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:16.363301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581c94b-8c69-48cc-a398-aa023b793fcc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:16.401016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e9c18-e219-416e-843a-598c0c181109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:16.401640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e9c18-e219-416e-843a-598c0c181109 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:16.401920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e9c18-e219-416e-843a-598c0c181109 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:16.402218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e9c18-e219-416e-843a-598c0c181109 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:16.405731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7b50e-ca5e-4bb6-8bb0-f34c9bb442ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:16.406603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7b50e-ca5e-4bb6-8bb0-f34c9bb442ec None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:16.407341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7b50e-ca5e-4bb6-8bb0-f34c9bb442ec None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:16.407823 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7b50e-ca5e-4bb6-8bb0-f34c9bb442ec None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:16.409124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7b50e-ca5e-4bb6-8bb0-f34c9bb442ec None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:16.410906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7b50e-ca5e-4bb6-8bb0-f34c9bb442ec None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:16.412545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e9c18-e219-416e-843a-598c0c181109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:16.413225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 218/423] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:16 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:16.414583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9469915-a6dd-4fee-a7aa-f5cb1845bf59 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:16.415082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9469915-a6dd-4fee-a7aa-f5cb1845bf59 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:16.415554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9469915-a6dd-4fee-a7aa-f5cb1845bf59 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:16.416924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 219/424] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:16.418084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9469915-a6dd-4fee-a7aa-f5cb1845bf59 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-905e9c18-e219-416e-843a-598c0c181109 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:16.422681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:16.423592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:16.424365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 206/425] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:16.425554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5905adf-ff84-4e47-a11b-72adf638cbc7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:16.425554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5905adf-ff84-4e47-a11b-72adf638cbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:16.425792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5905adf-ff84-4e47-a11b-72adf638cbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:16.444792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d5905adf-ff84-4e47-a11b-72adf638cbc7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:16.446204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d5905adf-ff84-4e47-a11b-72adf638cbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:16.522804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 207/426] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:16.530739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5905adf-ff84-4e47-a11b-72adf638cbc7 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:16.533639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9469915-a6dd-4fee-a7aa-f5cb1845bf59 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:16.535423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:16.570461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fc4f1-fdae-42e4-b542-5ecedd53a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:16.571477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fc4f1-fdae-42e4-b542-5ecedd53a7fd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:16.571950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fc4f1-fdae-42e4-b542-5ecedd53a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:16.572410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fc4f1-fdae-42e4-b542-5ecedd53a7fd None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:16.574240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fc4f1-fdae-42e4-b542-5ecedd53a7fd None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:16.576452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fc4f1-fdae-42e4-b542-5ecedd53a7fd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:16.578561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 220/427] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:16 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:16.594838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fc4f1-fdae-42e4-b542-5ecedd53a7fd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:16.704080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fc4f1-fdae-42e4-b542-5ecedd53a7fd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:16.711609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:16.744112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17470620-6d2f-428c-880c-1d847e7d81f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:16.745035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17470620-6d2f-428c-880c-1d847e7d81f1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:16.745527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17470620-6d2f-428c-880c-1d847e7d81f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:16.746025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17470620-6d2f-428c-880c-1d847e7d81f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:16.747074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17470620-6d2f-428c-880c-1d847e7d81f1 None admin] RBAC: Adding query filter params (group.id=5be88540f41f47679a98363c19a31fec, role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=ad635390dd3d4dcf897fa77044fd970e) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:16.748119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17470620-6d2f-428c-880c-1d847e7d81f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:16.748708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5905adf-ff84-4e47-a11b-72adf638cbc7 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:16.750012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17470620-6d2f-428c-880c-1d847e7d81f1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:16.756518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 221/428] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:16 2022] GET /identity/v3/role_assignments?group.id=5be88540f41f47679a98363c19a31fec&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:16.793434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfe5f0-205e-490b-a989-6505b5b12677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:16.794648 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfe5f0-205e-490b-a989-6505b5b12677 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:16.794934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfe5f0-205e-490b-a989-6505b5b12677 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:16.795206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfe5f0-205e-490b-a989-6505b5b12677 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:16.795881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfe5f0-205e-490b-a989-6505b5b12677 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:16.796180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfe5f0-205e-490b-a989-6505b5b12677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:16.797618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfe5f0-205e-490b-a989-6505b5b12677 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:16.807090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 208/429] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:16.813298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17470620-6d2f-428c-880c-1d847e7d81f1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:16.865973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:16.945730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfe5f0-205e-490b-a989-6505b5b12677 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:16.957212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17470620-6d2f-428c-880c-1d847e7d81f1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:16.997012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178b758-7f4d-454b-a8f7-3e0dcfa5557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:16.997910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b758-7f4d-454b-a8f7-3e0dcfa5557a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:16.998661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b758-7f4d-454b-a8f7-3e0dcfa5557a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:16.999245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b758-7f4d-454b-a8f7-3e0dcfa5557a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.075419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfe5f0-205e-490b-a989-6505b5b12677 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:17.089112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178b758-7f4d-454b-a8f7-3e0dcfa5557a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:17.090809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178b758-7f4d-454b-a8f7-3e0dcfa5557a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:17.125450 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f712e-42a0-44db-98e2-ea8b8d90291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:17.127116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f712e-42a0-44db-98e2-ea8b8d90291b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.127556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f712e-42a0-44db-98e2-ea8b8d90291b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.128227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f712e-42a0-44db-98e2-ea8b8d90291b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.136410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:17.151860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f712e-42a0-44db-98e2-ea8b8d90291b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-bf0f712e-42a0-44db-98e2-ea8b8d90291b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:17.158320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:17.159183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:17.159956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 209/430] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:17.166897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f712e-42a0-44db-98e2-ea8b8d90291b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:17.183586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:17.228535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 222/431] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:17.230485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178b758-7f4d-454b-a8f7-3e0dcfa5557a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:17.305617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f712e-42a0-44db-98e2-ea8b8d90291b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:17.341708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fdb6a-946c-4190-9224-10f09fe057c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:17.342632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fdb6a-946c-4190-9224-10f09fe057c9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.343109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fdb6a-946c-4190-9224-10f09fe057c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.343556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fdb6a-946c-4190-9224-10f09fe057c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.343902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178b758-7f4d-454b-a8f7-3e0dcfa5557a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:17.344569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fdb6a-946c-4190-9224-10f09fe057c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:17.345038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fdb6a-946c-4190-9224-10f09fe057c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:17.346602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fdb6a-946c-4190-9224-10f09fe057c9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:17.360231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 210/432] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:14:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:17.361714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fdb6a-946c-4190-9224-10f09fe057c9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:17.385658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49588733-618b-4ac2-8bc4-0072f951bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:17.386390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49588733-618b-4ac2-8bc4-0072f951bf8e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.386611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49588733-618b-4ac2-8bc4-0072f951bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.386890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49588733-618b-4ac2-8bc4-0072f951bf8e None admin] PATH_INFO: `/v3/projects/ad635390dd3d4dcf897fa77044fd970e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.388836 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49588733-618b-4ac2-8bc4-0072f951bf8e None admin] RBAC: Authorizing `identity:get_project(project_id=ad635390dd3d4dcf897fa77044fd970e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:17.391056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49588733-618b-4ac2-8bc4-0072f951bf8e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:17.395282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 223/433] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:17 2022] GET /identity/v3/projects/ad635390dd3d4dcf897fa77044fd970e => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:17.399500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49588733-618b-4ac2-8bc4-0072f951bf8e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:17.406225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:17.449705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:17.483345 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fdb6a-946c-4190-9224-10f09fe057c9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:17.509629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49588733-618b-4ac2-8bc4-0072f951bf8e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:17.522119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023437f-5ffe-4fa4-83f0-ce412a2e12b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:17.523188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023437f-5ffe-4fa4-83f0-ce412a2e12b2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.523725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023437f-5ffe-4fa4-83f0-ce412a2e12b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.524208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023437f-5ffe-4fa4-83f0-ce412a2e12b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.552924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf66e94-750c-481d-bb19-c06496e8040b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:17.553577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf66e94-750c-481d-bb19-c06496e8040b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.553902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf66e94-750c-481d-bb19-c06496e8040b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.554224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf66e94-750c-481d-bb19-c06496e8040b None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.555409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf66e94-750c-481d-bb19-c06496e8040b None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:17.557071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf66e94-750c-481d-bb19-c06496e8040b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:17.563851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 224/434] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:17 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:17.565202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf66e94-750c-481d-bb19-c06496e8040b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:17.603491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023437f-5ffe-4fa4-83f0-ce412a2e12b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:17.605261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023437f-5ffe-4fa4-83f0-ce412a2e12b2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:17.641311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:17.671591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf66e94-750c-481d-bb19-c06496e8040b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:17.676470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:17.706745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c36202-a94e-4094-bbb4-9e262d5d2b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:17.707334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c36202-a94e-4094-bbb4-9e262d5d2b95 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.707610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c36202-a94e-4094-bbb4-9e262d5d2b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.707880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c36202-a94e-4094-bbb4-9e262d5d2b95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.708019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 211/435] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 05:14:17.708587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c36202-a94e-4094-bbb4-9e262d5d2b95 None admin] RBAC: Adding query filter params (group.id=5be88540f41f47679a98363c19a31fec, role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=ad635390dd3d4dcf897fa77044fd970e) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:17.708892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c36202-a94e-4094-bbb4-9e262d5d2b95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:17.709267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a0152-4aa7-428a-8967-6dd06b92b478 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.709549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a0152-4aa7-428a-8967-6dd06b92b478 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.709819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a0152-4aa7-428a-8967-6dd06b92b478 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.710424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c36202-a94e-4094-bbb4-9e262d5d2b95 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:17.711212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 212/436] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:17.715140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b49eb9-b516-4edc-8c67-2008c07383b0 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.715683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b49eb9-b516-4edc-8c67-2008c07383b0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.715683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b49eb9-b516-4edc-8c67-2008c07383b0 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.717515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 213/437] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:17.718217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b49eb9-b516-4edc-8c67-2008c07383b0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:17.718646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 225/438] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:17 2022] GET /identity/v3/role_assignments?group.id=5be88540f41f47679a98363c19a31fec&scope.project.id=ad635390dd3d4dcf897fa77044fd970e&role.id=d36487377a664531a7f2e06f2f648a6f => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:17.719793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ce5da-4a03-4ebc-8e50-4254cd3aeddd None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.720082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ce5da-4a03-4ebc-8e50-4254cd3aeddd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.720356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ce5da-4a03-4ebc-8e50-4254cd3aeddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.739247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0a3ce5da-4a03-4ebc-8e50-4254cd3aeddd None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:17.740366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0a3ce5da-4a03-4ebc-8e50-4254cd3aeddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:17.819089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 226/439] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:17.820358 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24ecf-0fb4-4de7-b88f-5e893c9d98b0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.820642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24ecf-0fb4-4de7-b88f-5e893c9d98b0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.820912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24ecf-0fb4-4de7-b88f-5e893c9d98b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.840822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-05a24ecf-0fb4-4de7-b88f-5e893c9d98b0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:17.842127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-05a24ecf-0fb4-4de7-b88f-5e893c9d98b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:17.857079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b49eb9-b516-4edc-8c67-2008c07383b0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:17.907992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1fc46-1c00-4c6a-9a81-f573350bf4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:17.909245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1fc46-1c00-4c6a-9a81-f573350bf4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:17.909520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1fc46-1c00-4c6a-9a81-f573350bf4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:17.909789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1fc46-1c00-4c6a-9a81-f573350bf4b8 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:17.913222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1fc46-1c00-4c6a-9a81-f573350bf4b8 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:17.913333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1fc46-1c00-4c6a-9a81-f573350bf4b8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:17.914278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:17.917199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 214/440] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:17 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:17.921992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1fc46-1c00-4c6a-9a81-f573350bf4b8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:17.928351 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 227/441] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:17.968596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:18.035948 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1fc46-1c00-4c6a-9a81-f573350bf4b8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:18.072106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbd4b7-65b6-493e-b1f3-414542caf8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:18.073069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbd4b7-65b6-493e-b1f3-414542caf8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:18.073545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbd4b7-65b6-493e-b1f3-414542caf8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:18.074115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbd4b7-65b6-493e-b1f3-414542caf8b7 None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:18.076102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbd4b7-65b6-493e-b1f3-414542caf8b7 None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:18.077726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbd4b7-65b6-493e-b1f3-414542caf8b7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:18.079858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 215/442] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:17 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:18.085503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a24ecf-0fb4-4de7-b88f-5e893c9d98b0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:18.099835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbd4b7-65b6-493e-b1f3-414542caf8b7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:18.214643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a24ecf-0fb4-4de7-b88f-5e893c9d98b0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:18.214839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbd4b7-65b6-493e-b1f3-414542caf8b7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:18.224724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:18.253562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8892-1c59-4f2c-8fed-0e0976bbedea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:18.254257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8892-1c59-4f2c-8fed-0e0976bbedea None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:18.254538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8892-1c59-4f2c-8fed-0e0976bbedea None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:18.254811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8892-1c59-4f2c-8fed-0e0976bbedea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:18.257082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdbf34-4faf-4405-a1af-c7d9a6d22418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:18.258023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdbf34-4faf-4405-a1af-c7d9a6d22418 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:18.258316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdbf34-4faf-4405-a1af-c7d9a6d22418 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:18.258592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdbf34-4faf-4405-a1af-c7d9a6d22418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:18.259326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdbf34-4faf-4405-a1af-c7d9a6d22418 None admin] RBAC: Adding query filter params (group.id=5be88540f41f47679a98363c19a31fec, role.id=d36487377a664531a7f2e06f2f648a6f, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:18.259638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdbf34-4faf-4405-a1af-c7d9a6d22418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:18.261067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdbf34-4faf-4405-a1af-c7d9a6d22418 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:18.265341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a8892-1c59-4f2c-8fed-0e0976bbedea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:18.267314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 228/443] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/role_assignments?group.id=5be88540f41f47679a98363c19a31fec&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=d36487377a664531a7f2e06f2f648a6f => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:18.267877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdbf34-4faf-4405-a1af-c7d9a6d22418 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-436a8892-1c59-4f2c-8fed-0e0976bbedea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:18.271105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:18.271989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:18.273718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 216/444] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:18.279289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8892-1c59-4f2c-8fed-0e0976bbedea None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:18.357807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:18.411113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8892-1c59-4f2c-8fed-0e0976bbedea None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:18.411812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdbf34-4faf-4405-a1af-c7d9a6d22418 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:18.457135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924693-45be-448e-bfb0-6c6f146fbfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:18.457784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924693-45be-448e-bfb0-6c6f146fbfa9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:18.458126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924693-45be-448e-bfb0-6c6f146fbfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:18.458393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924693-45be-448e-bfb0-6c6f146fbfa9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:18.467874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd622218-16db-4bbe-b956-3540c9dcef58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:18.468795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd622218-16db-4bbe-b956-3540c9dcef58 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:18.469280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd622218-16db-4bbe-b956-3540c9dcef58 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:18.469766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd622218-16db-4bbe-b956-3540c9dcef58 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:18.471420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd622218-16db-4bbe-b956-3540c9dcef58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:18.471946 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd622218-16db-4bbe-b956-3540c9dcef58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:18.473825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f924693-45be-448e-bfb0-6c6f146fbfa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:18.474378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd622218-16db-4bbe-b956-3540c9dcef58 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-8f924693-45be-448e-bfb0-6c6f146fbfa9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:18.481269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:18.482965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:18.483787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 229/445] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:18.488107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 217/446] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:18.491024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924693-45be-448e-bfb0-6c6f146fbfa9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:18.494118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd622218-16db-4bbe-b956-3540c9dcef58 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:18.530610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:18.620380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd622218-16db-4bbe-b956-3540c9dcef58 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:18.621501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924693-45be-448e-bfb0-6c6f146fbfa9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:18.645973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:18.663237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e9b51-ef2e-4909-8a8b-6fdbe6a53288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:18.663855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e9b51-ef2e-4909-8a8b-6fdbe6a53288 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:18.664133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e9b51-ef2e-4909-8a8b-6fdbe6a53288 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:18.664406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e9b51-ef2e-4909-8a8b-6fdbe6a53288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:18.667772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107c43c-417c-4a30-b5f9-f123a7312a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:18.668382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107c43c-417c-4a30-b5f9-f123a7312a7c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:18.668763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107c43c-417c-4a30-b5f9-f123a7312a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:18.669124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107c43c-417c-4a30-b5f9-f123a7312a7c None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:18.670433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107c43c-417c-4a30-b5f9-f123a7312a7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:18.670740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107c43c-417c-4a30-b5f9-f123a7312a7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:18.672103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107c43c-417c-4a30-b5f9-f123a7312a7c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:18.682001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 230/447] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:18.686916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107c43c-417c-4a30-b5f9-f123a7312a7c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:18.738359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e9b51-ef2e-4909-8a8b-6fdbe6a53288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:18.739832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e9b51-ef2e-4909-8a8b-6fdbe6a53288 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:18.767840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:18.785382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107c43c-417c-4a30-b5f9-f123a7312a7c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:18.817919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4eebf7-e3e2-4da0-97e8-95987d9da9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:18.818539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4eebf7-e3e2-4da0-97e8-95987d9da9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:18.818826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4eebf7-e3e2-4da0-97e8-95987d9da9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:18.819089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4eebf7-e3e2-4da0-97e8-95987d9da9f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:18.835354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 218/448] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 05:14:18.840890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e9b51-ef2e-4909-8a8b-6fdbe6a53288 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:18.867805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:18.887592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4eebf7-e3e2-4da0-97e8-95987d9da9f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:18.889482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4eebf7-e3e2-4da0-97e8-95987d9da9f2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:18.935260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e9b51-ef2e-4909-8a8b-6fdbe6a53288 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:18.963442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:18.970308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d278102-0502-481e-bb72-eb8df7948836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:18.970927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d278102-0502-481e-bb72-eb8df7948836 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:18.971203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d278102-0502-481e-bb72-eb8df7948836 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:18.971472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d278102-0502-481e-bb72-eb8df7948836 None admin] PATH_INFO: `/v3/projects/01b2fa2971c94d418f0122cbda3c72f9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:18.972617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d278102-0502-481e-bb72-eb8df7948836 None admin] RBAC: Authorizing `identity:get_project(project_id=01b2fa2971c94d418f0122cbda3c72f9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:18.974756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d278102-0502-481e-bb72-eb8df7948836 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:18.976541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 219/449] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/projects/01b2fa2971c94d418f0122cbda3c72f9 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:18.982314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d278102-0502-481e-bb72-eb8df7948836 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:18.987082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 231/450] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:18.993680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4eebf7-e3e2-4da0-97e8-95987d9da9f2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:19.073748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:19.082299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d278102-0502-481e-bb72-eb8df7948836 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:19.093966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4eebf7-e3e2-4da0-97e8-95987d9da9f2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:19.115776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9ee82-56b3-4b57-8541-6e3fc073d2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:19.116398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9ee82-56b3-4b57-8541-6e3fc073d2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:19.116680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9ee82-56b3-4b57-8541-6e3fc073d2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:19.117184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9ee82-56b3-4b57-8541-6e3fc073d2c8 None admin] PATH_INFO: `/v3/groups/a3124b7e11ef4cc2b9c7f0b2bb19d19b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:19.118086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9ee82-56b3-4b57-8541-6e3fc073d2c8 None admin] RBAC: Authorizing `identity:get_group(group_id=a3124b7e11ef4cc2b9c7f0b2bb19d19b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:19.119526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9ee82-56b3-4b57-8541-6e3fc073d2c8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:19.121076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 220/451] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/groups/a3124b7e11ef4cc2b9c7f0b2bb19d19b => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:19.126956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9ee82-56b3-4b57-8541-6e3fc073d2c8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:19.127076 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd8ae6-26ac-47a4-9b76-8c6c36667369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:19.127575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd8ae6-26ac-47a4-9b76-8c6c36667369 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:19.127850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd8ae6-26ac-47a4-9b76-8c6c36667369 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:19.128119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd8ae6-26ac-47a4-9b76-8c6c36667369 None admin] PATH_INFO: `/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:19.129339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd8ae6-26ac-47a4-9b76-8c6c36667369 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7cbe24d21e4e6aa95e64e0ae6dfba2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:19.130654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd8ae6-26ac-47a4-9b76-8c6c36667369 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:19.132230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 232/452] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:14:18 2022] GET /identity/v3/projects/fe7cbe24d21e4e6aa95e64e0ae6dfba2 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:19.137328 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd8ae6-26ac-47a4-9b76-8c6c36667369 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:19.189756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:19.223918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9ee82-56b3-4b57-8541-6e3fc073d2c8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:19.234715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd8ae6-26ac-47a4-9b76-8c6c36667369 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:19.256746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3caa8-fb8e-49d2-8b64-61d8c1b98c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:19.257341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3caa8-fb8e-49d2-8b64-61d8c1b98c52 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:19.257620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3caa8-fb8e-49d2-8b64-61d8c1b98c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:19.257892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3caa8-fb8e-49d2-8b64-61d8c1b98c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:19.258638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3caa8-fb8e-49d2-8b64-61d8c1b98c52 None admin] RBAC: Adding query filter params (group.id=a3124b7e11ef4cc2b9c7f0b2bb19d19b, role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=01b2fa2971c94d418f0122cbda3c72f9) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:19.258951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3caa8-fb8e-49d2-8b64-61d8c1b98c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:19.260391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3caa8-fb8e-49d2-8b64-61d8c1b98c52 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:19.266984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 221/453] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:19 2022] GET /identity/v3/role_assignments?group.id=a3124b7e11ef4cc2b9c7f0b2bb19d19b&scope.project.id=01b2fa2971c94d418f0122cbda3c72f9&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:19.272740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd249e1f-9cc1-425c-9e32-4755f32769ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:19.273394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd249e1f-9cc1-425c-9e32-4755f32769ba None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:19.273677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd249e1f-9cc1-425c-9e32-4755f32769ba None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:19.273971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd249e1f-9cc1-425c-9e32-4755f32769ba None admin] PATH_INFO: `/v3/groups/5be88540f41f47679a98363c19a31fec` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:19.275136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd249e1f-9cc1-425c-9e32-4755f32769ba None admin] RBAC: Authorizing `identity:get_group(group_id=5be88540f41f47679a98363c19a31fec)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:19.276597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd249e1f-9cc1-425c-9e32-4755f32769ba None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:19.278292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 233/454] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Sep 21 05:14:19 2022] GET /identity/v3/groups/5be88540f41f47679a98363c19a31fec => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:19.284404 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3caa8-fb8e-49d2-8b64-61d8c1b98c52 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:19.394730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3caa8-fb8e-49d2-8b64-61d8c1b98c52 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:19.424180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:19.431584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bd662-ba2e-466b-bd32-fd16e1f21c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:19.432196 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bd662-ba2e-466b-bd32-fd16e1f21c5b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:19.432478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bd662-ba2e-466b-bd32-fd16e1f21c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:19.432753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bd662-ba2e-466b-bd32-fd16e1f21c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:19.433518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bd662-ba2e-466b-bd32-fd16e1f21c5b None admin] RBAC: Adding query filter params (group.id=5be88540f41f47679a98363c19a31fec, role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:19.433833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bd662-ba2e-466b-bd32-fd16e1f21c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:19.435844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bd662-ba2e-466b-bd32-fd16e1f21c5b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:19.441819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 222/455] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Sep 21 05:14:19 2022] GET /identity/v3/role_assignments?group.id=5be88540f41f47679a98363c19a31fec&scope.project.id=fe7cbe24d21e4e6aa95e64e0ae6dfba2&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:22.183954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c651f2-f3b3-49d7-abd5-5efad558008e None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:22.184506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c651f2-f3b3-49d7-abd5-5efad558008e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:22.184506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c651f2-f3b3-49d7-abd5-5efad558008e None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:22.186235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 234/456] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:22.191359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b2b69-7194-4135-ad89-e695add0fdf0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:22.191760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b2b69-7194-4135-ad89-e695add0fdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:22.191955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b2b69-7194-4135-ad89-e695add0fdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:22.226981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d25b2b69-7194-4135-ad89-e695add0fdf0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:22.228208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d25b2b69-7194-4135-ad89-e695add0fdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:22.263820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da92ad1-96f6-40a1-a4cc-89510050d68a None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:22.264144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da92ad1-96f6-40a1-a4cc-89510050d68a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:22.264371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da92ad1-96f6-40a1-a4cc-89510050d68a None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:22.265950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 235/457] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:22.274382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610985e-029d-4be6-9c4e-e6d4488d5991 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:22.274856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610985e-029d-4be6-9c4e-e6d4488d5991 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:22.275278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610985e-029d-4be6-9c4e-e6d4488d5991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:22.307789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9610985e-029d-4be6-9c4e-e6d4488d5991 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:22.309253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9610985e-029d-4be6-9c4e-e6d4488d5991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:22.309430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 223/458] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:22.322610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:22.412261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 236/459] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:22.693965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b2b69-7194-4135-ad89-e695add0fdf0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:22.836149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3f720-ee1c-4c2d-ae4d-694a69e3779c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:22.836519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3f720-ee1c-4c2d-ae4d-694a69e3779c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:22.836696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3f720-ee1c-4c2d-ae4d-694a69e3779c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:22.839055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b2b69-7194-4135-ad89-e695add0fdf0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:22.842059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 237/460] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:22.854699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaca5b-4711-4776-a885-c9001762601f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:22.854989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaca5b-4711-4776-a885-c9001762601f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:22.855346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaca5b-4711-4776-a885-c9001762601f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:22.874788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-08aaca5b-4711-4776-a885-c9001762601f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:22.874958 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:22.876437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-08aaca5b-4711-4776-a885-c9001762601f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:22.893163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b72ec-cfdb-4591-ab1a-6a8560bc16f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:22.893809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b72ec-cfdb-4591-ab1a-6a8560bc16f7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:22.894120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b72ec-cfdb-4591-ab1a-6a8560bc16f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:22.894391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b72ec-cfdb-4591-ab1a-6a8560bc16f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:22.906314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b72ec-cfdb-4591-ab1a-6a8560bc16f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:22.908314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b72ec-cfdb-4591-ab1a-6a8560bc16f7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-9c9b72ec-cfdb-4591-ab1a-6a8560bc16f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:22.929301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:22.930294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:22.931128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:22.931128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:22.931128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:22.931128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 224/461] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:22.936735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b72ec-cfdb-4591-ab1a-6a8560bc16f7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:22.950905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 238/462] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:22.952142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aaca5b-4711-4776-a885-c9001762601f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:23.041467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:23.060961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b72ec-cfdb-4591-ab1a-6a8560bc16f7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:23.071945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aaca5b-4711-4776-a885-c9001762601f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:23.104621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8f4dc-08d6-4d0a-9990-660cd7b43922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:23.105296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8f4dc-08d6-4d0a-9990-660cd7b43922 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.105522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8f4dc-08d6-4d0a-9990-660cd7b43922 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.105790 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8f4dc-08d6-4d0a-9990-660cd7b43922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.119172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8f4dc-08d6-4d0a-9990-660cd7b43922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:23.120624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8f4dc-08d6-4d0a-9990-660cd7b43922 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:23.123511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdef38-bc7d-4c16-adc2-74fb0b19a68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:23.124344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdef38-bc7d-4c16-adc2-74fb0b19a68d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.124825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdef38-bc7d-4c16-adc2-74fb0b19a68d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.125329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdef38-bc7d-4c16-adc2-74fb0b19a68d None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.126240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdef38-bc7d-4c16-adc2-74fb0b19a68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:23.126750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdef38-bc7d-4c16-adc2-74fb0b19a68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:23.129339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdef38-bc7d-4c16-adc2-74fb0b19a68d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-b0a8f4dc-08d6-4d0a-9990-660cd7b43922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:23.135068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:23.135955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:23.136805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:23.136805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:23.136805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:23.141315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 225/463] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:23.142607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38de9b8-d9f9-470b-8d71-8f8520030fc8 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.142889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38de9b8-d9f9-470b-8d71-8f8520030fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.143235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38de9b8-d9f9-470b-8d71-8f8520030fc8 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.144553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 226/464] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:23.145395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38de9b8-d9f9-470b-8d71-8f8520030fc8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:23.145613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 239/465] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:23.154888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531aaf62-d14a-4e00-9c99-dc618073e23f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.154888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531aaf62-d14a-4e00-9c99-dc618073e23f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.154888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531aaf62-d14a-4e00-9c99-dc618073e23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.172369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-531aaf62-d14a-4e00-9c99-dc618073e23f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:23.173961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-531aaf62-d14a-4e00-9c99-dc618073e23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:23.180077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:23.253516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 240/466] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:14:23.255099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531aaf62-d14a-4e00-9c99-dc618073e23f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:23.273731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38de9b8-d9f9-470b-8d71-8f8520030fc8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:23.321633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0698ed8-5008-407d-8f3e-b881180e830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:23.322836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0698ed8-5008-407d-8f3e-b881180e830e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.323261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0698ed8-5008-407d-8f3e-b881180e830e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.323773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0698ed8-5008-407d-8f3e-b881180e830e None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.324553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0698ed8-5008-407d-8f3e-b881180e830e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:23.325770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:23.326563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0698ed8-5008-407d-8f3e-b881180e830e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:23.328751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0698ed8-5008-407d-8f3e-b881180e830e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:23.346802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 227/467] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:23.348157 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0698ed8-5008-407d-8f3e-b881180e830e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:23.365502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531aaf62-d14a-4e00-9c99-dc618073e23f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:23.401475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e7763-96ab-48a2-baf4-04f57692d779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:23.402311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e7763-96ab-48a2-baf4-04f57692d779 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.402690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e7763-96ab-48a2-baf4-04f57692d779 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.403064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e7763-96ab-48a2-baf4-04f57692d779 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.404430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e7763-96ab-48a2-baf4-04f57692d779 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:23.406056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e7763-96ab-48a2-baf4-04f57692d779 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:23.433354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 241/468] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:14:23 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:14:23.434469 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e7763-96ab-48a2-baf4-04f57692d779 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:23.465128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:23.479614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0698ed8-5008-407d-8f3e-b881180e830e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:23.518622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bcb06-4e60-446b-8b4f-96eb8db6f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:23.519419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bcb06-4e60-446b-8b4f-96eb8db6f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.519841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bcb06-4e60-446b-8b4f-96eb8db6f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.520453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bcb06-4e60-446b-8b4f-96eb8db6f3f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.532138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bcb06-4e60-446b-8b4f-96eb8db6f3f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:23.533981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bcb06-4e60-446b-8b4f-96eb8db6f3f5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-a67bcb06-4e60-446b-8b4f-96eb8db6f3f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:23.551220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:23.552358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:23.553175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:23.553175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:23.553175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:23.553793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 228/469] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:23.558097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bcb06-4e60-446b-8b4f-96eb8db6f3f5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:23.580424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e7763-96ab-48a2-baf4-04f57692d779 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:23.610075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:23.637121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5806a2f-c7b2-45a7-af1d-dc5f7de86fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:23.638924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5806a2f-c7b2-45a7-af1d-dc5f7de86fce None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.639422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5806a2f-c7b2-45a7-af1d-dc5f7de86fce None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.640148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5806a2f-c7b2-45a7-af1d-dc5f7de86fce None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.641516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5806a2f-c7b2-45a7-af1d-dc5f7de86fce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:23.643881 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5806a2f-c7b2-45a7-af1d-dc5f7de86fce None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:23.676910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 242/470] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:14:23 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:14:23.678293 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5806a2f-c7b2-45a7-af1d-dc5f7de86fce None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:23.701094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bcb06-4e60-446b-8b4f-96eb8db6f3f5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:23.749042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37731d8-62d3-4cb3-bea3-5ac73f90f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:23.749848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37731d8-62d3-4cb3-bea3-5ac73f90f379 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.750264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37731d8-62d3-4cb3-bea3-5ac73f90f379 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.750557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37731d8-62d3-4cb3-bea3-5ac73f90f379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.764437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:23.767336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37731d8-62d3-4cb3-bea3-5ac73f90f379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:23.769037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37731d8-62d3-4cb3-bea3-5ac73f90f379 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-b37731d8-62d3-4cb3-bea3-5ac73f90f379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:23.787301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:23.788289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:23.789112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:23.789112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:23.789112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:23.789804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 229/471] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:23.797675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37731d8-62d3-4cb3-bea3-5ac73f90f379 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:23.837468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5806a2f-c7b2-45a7-af1d-dc5f7de86fce None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:23.892687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465ed78-2ac1-4527-a4ee-6333e72a4194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:23.893235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465ed78-2ac1-4527-a4ee-6333e72a4194 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:23.893513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465ed78-2ac1-4527-a4ee-6333e72a4194 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:23.893783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465ed78-2ac1-4527-a4ee-6333e72a4194 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:23.894451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465ed78-2ac1-4527-a4ee-6333e72a4194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:23.894757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465ed78-2ac1-4527-a4ee-6333e72a4194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:23.896425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465ed78-2ac1-4527-a4ee-6333e72a4194 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:23.916824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 243/472] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:23.922344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465ed78-2ac1-4527-a4ee-6333e72a4194 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:23.955556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37731d8-62d3-4cb3-bea3-5ac73f90f379 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:23.977914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:24.002105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ccee1-db14-4674-b207-3330d13ad3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:24.002715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ccee1-db14-4674-b207-3330d13ad3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:24.003057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ccee1-db14-4674-b207-3330d13ad3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:24.003329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ccee1-db14-4674-b207-3330d13ad3f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:24.004024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ccee1-db14-4674-b207-3330d13ad3f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:24.004334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ccee1-db14-4674-b207-3330d13ad3f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:24.005719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ccee1-db14-4674-b207-3330d13ad3f3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:24.023168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 230/473] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:24.024429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5a2be-44c5-4435-8f95-9293c10b3441 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:24.024714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5a2be-44c5-4435-8f95-9293c10b3441 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:24.024985 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5a2be-44c5-4435-8f95-9293c10b3441 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:24.030688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 231/474] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:24.031646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5a2be-44c5-4435-8f95-9293c10b3441 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:24.056116 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465ed78-2ac1-4527-a4ee-6333e72a4194 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:24.097476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e85f8-c1bc-4a7a-90a2-f8f50af83d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:24.098463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e85f8-c1bc-4a7a-90a2-f8f50af83d12 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:24.098962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e85f8-c1bc-4a7a-90a2-f8f50af83d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:24.099901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:24.101065 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e85f8-c1bc-4a7a-90a2-f8f50af83d12 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:24.102359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e85f8-c1bc-4a7a-90a2-f8f50af83d12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:24.104000 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e85f8-c1bc-4a7a-90a2-f8f50af83d12 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:24.132924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 244/475] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:14:23 2022] POST /identity/v3/users => generated 249 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:14:24.135519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc7b87-4622-4cee-ad68-64869a3b2060 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:24.136340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc7b87-4622-4cee-ad68-64869a3b2060 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:24.136859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc7b87-4622-4cee-ad68-64869a3b2060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:24.158207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5a2be-44c5-4435-8f95-9293c10b3441 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:24.159608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6bdc7b87-4622-4cee-ad68-64869a3b2060 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:24.161237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6bdc7b87-4622-4cee-ad68-64869a3b2060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:24.204495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2a4b2-7a3e-49b6-888e-5f863445710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:24.205337 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2a4b2-7a3e-49b6-888e-5f863445710a None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:24.206405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2a4b2-7a3e-49b6-888e-5f863445710a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:24.206686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2a4b2-7a3e-49b6-888e-5f863445710a None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:24.208438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2a4b2-7a3e-49b6-888e-5f863445710a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:24.211869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2a4b2-7a3e-49b6-888e-5f863445710a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:24.255428 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 232/476] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:14:24 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:14:24.270661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 245/477] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:24.559686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2a4b2-7a3e-49b6-888e-5f863445710a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:24.694744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2a4b2-7a3e-49b6-888e-5f863445710a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:24.705027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:24.742827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:24.743802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:24.744277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:24.744746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:24.745627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:24.747254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:24.764086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.sql.core [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 21 05:14:24.764086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 21 05:14:24.764086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [parameters: {'id': 'cdf6631ecd844346846e214e8f1954ae', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 21 05:14:24.764086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81411) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Sep 21 05:14:24.785611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return ref.to_dict() Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 05:14:24.786560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application yield resource Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.rollback() Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.force_reraise() Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise self.value Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application session.commit() Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 21 05:14:24.787536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._prepare_impl() Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.flush() Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._flush(objects) Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application compat.raise_( Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application flush_context.execute() Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rec.execute(self) Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 21 05:14:24.788358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 05:14:24.789239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 05:14:24.795709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [parameters: {'id': 'cdf6631ecd844346846e214e8f1954ae', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:24.798214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 21 05:14:24.800687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:24.802847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 233/478] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Sep 21 05:14:24 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:24.804210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc7b87-4622-4cee-ad68-64869a3b2060 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:24.852423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:24.954145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc7b87-4622-4cee-ad68-64869a3b2060 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:24.988644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113dd6b-df1a-4ce2-aa1e-a761a9ea9633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:24.989276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113dd6b-df1a-4ce2-aa1e-a761a9ea9633 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:24.989556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113dd6b-df1a-4ce2-aa1e-a761a9ea9633 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:24.989830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113dd6b-df1a-4ce2-aa1e-a761a9ea9633 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:25.000229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113dd6b-df1a-4ce2-aa1e-a761a9ea9633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f113dd6b-df1a-4ce2-aa1e-a761a9ea9633 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:25.005098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:25.007197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:25.009175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 246/479] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:14:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:25.012386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:25.146146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034661b-156e-407d-b720-e3dcbb7e6204 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:25.182144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a08d43-e187-40fb-9a20-0f4685574f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:25.183075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a08d43-e187-40fb-9a20-0f4685574f2b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:25.183545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a08d43-e187-40fb-9a20-0f4685574f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:25.183994 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a08d43-e187-40fb-9a20-0f4685574f2b None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:25.184921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a08d43-e187-40fb-9a20-0f4685574f2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:25.185404 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a08d43-e187-40fb-9a20-0f4685574f2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:25.186995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a08d43-e187-40fb-9a20-0f4685574f2b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:25.196105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 234/480] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:26.108642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf83d5-7fd9-424a-98b6-6066139ab74b None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:26.109157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf83d5-7fd9-424a-98b6-6066139ab74b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:26.109256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf83d5-7fd9-424a-98b6-6066139ab74b None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:26.110787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 247/481] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:26.116229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f0b5-6d6e-4deb-8d68-294766a4719d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:26.116614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f0b5-6d6e-4deb-8d68-294766a4719d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:26.116775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f0b5-6d6e-4deb-8d68-294766a4719d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:26.139271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f067f0b5-6d6e-4deb-8d68-294766a4719d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:26.140460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f067f0b5-6d6e-4deb-8d68-294766a4719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:26.182465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:26.235502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 235/482] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:26.356675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcab5a6-b1a6-47f1-a11b-5e5d81339f90 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:26.357009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcab5a6-b1a6-47f1-a11b-5e5d81339f90 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:26.357228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcab5a6-b1a6-47f1-a11b-5e5d81339f90 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:26.358901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 248/483] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:26.363934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f7c3b-4870-429a-9422-e353d285e17f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:26.364433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f7c3b-4870-429a-9422-e353d285e17f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:26.364901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f7c3b-4870-429a-9422-e353d285e17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:26.385625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5f1f7c3b-4870-429a-9422-e353d285e17f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:26.386829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f1f7c3b-4870-429a-9422-e353d285e17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:26.484570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 236/484] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:26.653054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcab5a6-b1a6-47f1-a11b-5e5d81339f90 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:26.765304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:26.781671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcab5a6-b1a6-47f1-a11b-5e5d81339f90 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:26.814130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f7c3b-4870-429a-9422-e353d285e17f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:26.815651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad06bf1-1169-4bdd-a4da-ac434c4e11a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:26.816370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad06bf1-1169-4bdd-a4da-ac434c4e11a8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:26.816726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad06bf1-1169-4bdd-a4da-ac434c4e11a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:26.817043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad06bf1-1169-4bdd-a4da-ac434c4e11a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:26.840303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad06bf1-1169-4bdd-a4da-ac434c4e11a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5ad06bf1-1169-4bdd-a4da-ac434c4e11a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:26.845359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:26.846277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:26.847056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 249/485] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:26.852664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad06bf1-1169-4bdd-a4da-ac434c4e11a8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:26.984532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:26.997551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f7c3b-4870-429a-9422-e353d285e17f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.017078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad06bf1-1169-4bdd-a4da-ac434c4e11a8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.040735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661fac-4a6e-410d-ac7f-b5b681d12bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.041366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661fac-4a6e-410d-ac7f-b5b681d12bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.041644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661fac-4a6e-410d-ac7f-b5b681d12bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.041913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661fac-4a6e-410d-ac7f-b5b681d12bc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.053296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22661fac-4a6e-410d-ac7f-b5b681d12bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-22661fac-4a6e-410d-ac7f-b5b681d12bc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:27.058515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:27.061086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:27.062860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 237/486] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:27.069970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661fac-4a6e-410d-ac7f-b5b681d12bc4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:27.092963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e092-fd83-4c27-a360-0ef2b5e0acf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.093574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e092-fd83-4c27-a360-0ef2b5e0acf9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.093846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e092-fd83-4c27-a360-0ef2b5e0acf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.094139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e092-fd83-4c27-a360-0ef2b5e0acf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.095671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e092-fd83-4c27-a360-0ef2b5e0acf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:27.096641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e092-fd83-4c27-a360-0ef2b5e0acf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:27.098030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e092-fd83-4c27-a360-0ef2b5e0acf9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:27.107639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 250/487] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:27.113561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e092-fd83-4c27-a360-0ef2b5e0acf9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:27.123583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:27.225362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e092-fd83-4c27-a360-0ef2b5e0acf9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.232349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661fac-4a6e-410d-ac7f-b5b681d12bc4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.261228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c94c28-11d4-48a1-9982-a0423bca7cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.261777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c94c28-11d4-48a1-9982-a0423bca7cdf None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.262102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c94c28-11d4-48a1-9982-a0423bca7cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.262370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c94c28-11d4-48a1-9982-a0423bca7cdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.273307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c94c28-11d4-48a1-9982-a0423bca7cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:27.274777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c94c28-11d4-48a1-9982-a0423bca7cdf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:27.282773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5767-eedf-425c-a7c4-08d5fd470e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.283895 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5767-eedf-425c-a7c4-08d5fd470e28 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.284180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5767-eedf-425c-a7c4-08d5fd470e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.284453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5767-eedf-425c-a7c4-08d5fd470e28 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.285401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba5767-eedf-425c-a7c4-08d5fd470e28 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:27.285710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba5767-eedf-425c-a7c4-08d5fd470e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:27.287126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba5767-eedf-425c-a7c4-08d5fd470e28 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-34c94c28-11d4-48a1-9982-a0423bca7cdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:27.289824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:27.291847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:27.294451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:27.294451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:27.294451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:27.301956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 251/488] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:27.302164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 238/489] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:27.302332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d55d0c-5cc3-4eb5-96a4-cc1bd9a4844e None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.302332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d55d0c-5cc3-4eb5-96a4-cc1bd9a4844e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.302332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d55d0c-5cc3-4eb5-96a4-cc1bd9a4844e None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.302332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 252/490] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:27.302880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5767-eedf-425c-a7c4-08d5fd470e28 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:27.303030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d55d0c-5cc3-4eb5-96a4-cc1bd9a4844e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:27.324959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:27.371052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:27.416084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5767-eedf-425c-a7c4-08d5fd470e28 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.416437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d55d0c-5cc3-4eb5-96a4-cc1bd9a4844e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.451763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44626377-9cc0-4035-9669-ce4c1d19cf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.452439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44626377-9cc0-4035-9669-ce4c1d19cf10 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.452719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44626377-9cc0-4035-9669-ce4c1d19cf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.452989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44626377-9cc0-4035-9669-ce4c1d19cf10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.453528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f7c09-5d4e-44e9-a600-6c862ae80c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.454390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f7c09-5d4e-44e9-a600-6c862ae80c57 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.454970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f7c09-5d4e-44e9-a600-6c862ae80c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.455565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f7c09-5d4e-44e9-a600-6c862ae80c57 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.456276 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f7c09-5d4e-44e9-a600-6c862ae80c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:27.456587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f7c09-5d4e-44e9-a600-6c862ae80c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:27.458021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f7c09-5d4e-44e9-a600-6c862ae80c57 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:27.463544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44626377-9cc0-4035-9669-ce4c1d19cf10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:27.464961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44626377-9cc0-4035-9669-ce4c1d19cf10 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:27.475069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 239/491] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:27.475069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cddeb9-db3a-4b15-9de1-467113eea5e5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.475069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cddeb9-db3a-4b15-9de1-467113eea5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.475069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cddeb9-db3a-4b15-9de1-467113eea5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-44626377-9cc0-4035-9669-ce4c1d19cf10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:27.484260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:27.485121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:27.485972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:27.485972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:27.485972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:27.485972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 253/492] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:27.487018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578d887-df64-440e-9bb6-0f543e4bf840 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.487287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578d887-df64-440e-9bb6-0f543e4bf840 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.487558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578d887-df64-440e-9bb6-0f543e4bf840 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.489529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 254/493] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:27.489966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578d887-df64-440e-9bb6-0f543e4bf840 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:27.494476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-52cddeb9-db3a-4b15-9de1-467113eea5e5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:27.495669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-52cddeb9-db3a-4b15-9de1-467113eea5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:27.566340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 240/494] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:27.567562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cddeb9-db3a-4b15-9de1-467113eea5e5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:27.588994 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:27.601127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578d887-df64-440e-9bb6-0f543e4bf840 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.613838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:27.639703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c549026-c067-4fb3-9c63-0344cc575962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.640545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c549026-c067-4fb3-9c63-0344cc575962 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.640962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c549026-c067-4fb3-9c63-0344cc575962 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.641509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c549026-c067-4fb3-9c63-0344cc575962 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.649996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c549026-c067-4fb3-9c63-0344cc575962 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:27.651434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c549026-c067-4fb3-9c63-0344cc575962 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-0c549026-c067-4fb3-9c63-0344cc575962 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:27.666158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:27.667102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:27.668081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 255/495] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Sep 21 05:14:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:27.668632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bbd75-3a57-4e04-96cd-6b3d73576a7e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.668632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bbd75-3a57-4e04-96cd-6b3d73576a7e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.670216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bbd75-3a57-4e04-96cd-6b3d73576a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.694965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-203bbd75-3a57-4e04-96cd-6b3d73576a7e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:27.696782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-203bbd75-3a57-4e04-96cd-6b3d73576a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:27.716208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cddeb9-db3a-4b15-9de1-467113eea5e5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.752993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58184-4231-465e-bca1-1cfe71baed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.753657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58184-4231-465e-bca1-1cfe71baed50 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.754061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58184-4231-465e-bca1-1cfe71baed50 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.754334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58184-4231-465e-bca1-1cfe71baed50 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.754963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd58184-4231-465e-bca1-1cfe71baed50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:27.755257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd58184-4231-465e-bca1-1cfe71baed50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:27.756622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd58184-4231-465e-bca1-1cfe71baed50 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:27.770089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 241/496] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:27.770888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58184-4231-465e-bca1-1cfe71baed50 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:27.781081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 256/497] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:27.782199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bbd75-3a57-4e04-96cd-6b3d73576a7e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:27.866379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:27.904719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58184-4231-465e-bca1-1cfe71baed50 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.936440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:27.940515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bbd75-3a57-4e04-96cd-6b3d73576a7e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:27.955726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405875d-ed19-429b-9c71-23d3a760e407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.956879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405875d-ed19-429b-9c71-23d3a760e407 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.957300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405875d-ed19-429b-9c71-23d3a760e407 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.957714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405875d-ed19-429b-9c71-23d3a760e407 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.959293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405875d-ed19-429b-9c71-23d3a760e407 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:27.959751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405875d-ed19-429b-9c71-23d3a760e407 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:27.962157 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405875d-ed19-429b-9c71-23d3a760e407 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:27.985439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdf781-9a3f-4c34-935e-a4c4e4d9535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:27.986360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdf781-9a3f-4c34-935e-a4c4e4d9535b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:27.986780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdf781-9a3f-4c34-935e-a4c4e4d9535b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:27.987238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdf781-9a3f-4c34-935e-a4c4e4d9535b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:27.995203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdf781-9a3f-4c34-935e-a4c4e4d9535b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:27.996816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdf781-9a3f-4c34-935e-a4c4e4d9535b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:28.001260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 242/498] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Sep 21 05:14:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:28.002178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405875d-ed19-429b-9c71-23d3a760e407 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a1fdf781-9a3f-4c34-935e-a4c4e4d9535b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:28.010409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:28.011429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 21 05:14:28.012300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:28.015422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 257/499] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:28.016715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdf781-9a3f-4c34-935e-a4c4e4d9535b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.107522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405875d-ed19-429b-9c71-23d3a760e407 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:28.122288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdf781-9a3f-4c34-935e-a4c4e4d9535b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:28.130214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:28.142739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbac38-16bf-4076-a828-eeeb321efb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:28.143358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbac38-16bf-4076-a828-eeeb321efb96 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.143632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbac38-16bf-4076-a828-eeeb321efb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.143900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbac38-16bf-4076-a828-eeeb321efb96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.154386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbac38-16bf-4076-a828-eeeb321efb96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:28.155699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a874ebb-0e96-4940-927c-33d3cc222aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:28.156300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a874ebb-0e96-4940-927c-33d3cc222aeb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.156579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a874ebb-0e96-4940-927c-33d3cc222aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.156853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a874ebb-0e96-4940-927c-33d3cc222aeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-43fbac38-16bf-4076-a828-eeeb321efb96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:28.159241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:28.160296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 243/500] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:28.161105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbac38-16bf-4076-a828-eeeb321efb96 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.187706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:28.228248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a874ebb-0e96-4940-927c-33d3cc222aeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:28.229686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a874ebb-0e96-4940-927c-33d3cc222aeb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:28.256731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbac38-16bf-4076-a828-eeeb321efb96 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:28.289537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c615e7c-870c-46c9-893a-57a63c9bf252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:28.290158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c615e7c-870c-46c9-893a-57a63c9bf252 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.290436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c615e7c-870c-46c9-893a-57a63c9bf252 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.290702 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c615e7c-870c-46c9-893a-57a63c9bf252 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.291416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c615e7c-870c-46c9-893a-57a63c9bf252 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:28.291713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c615e7c-870c-46c9-893a-57a63c9bf252 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:28.293136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c615e7c-870c-46c9-893a-57a63c9bf252 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:28.310060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 244/501] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:28.311158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216511-0098-4d58-81c6-31c33469d63a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.311448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216511-0098-4d58-81c6-31c33469d63a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.311718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216511-0098-4d58-81c6-31c33469d63a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.313022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 245/502] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:28.313733 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7216511-0098-4d58-81c6-31c33469d63a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.332155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 258/503] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:28.332989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a874ebb-0e96-4940-927c-33d3cc222aeb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.360785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:28.393051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:28.419762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7216511-0098-4d58-81c6-31c33469d63a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:28.434216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a874ebb-0e96-4940-927c-33d3cc222aeb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:28.454039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39a9cb-39c7-4a2e-82bc-638fbd3b7fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:28.454658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39a9cb-39c7-4a2e-82bc-638fbd3b7fbd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.454930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39a9cb-39c7-4a2e-82bc-638fbd3b7fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.455198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39a9cb-39c7-4a2e-82bc-638fbd3b7fbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.465103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39a9cb-39c7-4a2e-82bc-638fbd3b7fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:28.467662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf04c9d-94ca-472f-9826-e4adced0e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:28.468261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf04c9d-94ca-472f-9826-e4adced0e4bb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.468541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf04c9d-94ca-472f-9826-e4adced0e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.468811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf04c9d-94ca-472f-9826-e4adced0e4bb None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.469458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf04c9d-94ca-472f-9826-e4adced0e4bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:28.469760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf04c9d-94ca-472f-9826-e4adced0e4bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-7b39a9cb-39c7-4a2e-82bc-638fbd3b7fbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:28.469904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:28.470953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:28.471797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:28.472457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf04c9d-94ca-472f-9826-e4adced0e4bb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:28.475818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 246/504] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:28.476678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39a9cb-39c7-4a2e-82bc-638fbd3b7fbd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.479412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 259/505] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:28.480736 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073e527-0b27-4bbc-868c-d90929d328a9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.481026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073e527-0b27-4bbc-868c-d90929d328a9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.481355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073e527-0b27-4bbc-868c-d90929d328a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.500253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d073e527-0b27-4bbc-868c-d90929d328a9 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:28.502409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d073e527-0b27-4bbc-868c-d90929d328a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:28.568767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 260/506] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:28.569587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073e527-0b27-4bbc-868c-d90929d328a9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.574070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39a9cb-39c7-4a2e-82bc-638fbd3b7fbd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:28.576264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:28.616256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c4b26-a4f0-4404-9115-529f6ec01d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:28.616861 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c4b26-a4f0-4404-9115-529f6ec01d60 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.617140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c4b26-a4f0-4404-9115-529f6ec01d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.617412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c4b26-a4f0-4404-9115-529f6ec01d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.631837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:28.684084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073e527-0b27-4bbc-868c-d90929d328a9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:28.688141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c4b26-a4f0-4404-9115-529f6ec01d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:28.689615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c4b26-a4f0-4404-9115-529f6ec01d60 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:28.719279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185bc02-efd3-4b89-95be-6dd407d6785c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:28.719912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185bc02-efd3-4b89-95be-6dd407d6785c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.720194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185bc02-efd3-4b89-95be-6dd407d6785c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.720470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185bc02-efd3-4b89-95be-6dd407d6785c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.730888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185bc02-efd3-4b89-95be-6dd407d6785c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:28.733036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185bc02-efd3-4b89-95be-6dd407d6785c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b185bc02-efd3-4b89-95be-6dd407d6785c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:28.749499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:28.750373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:28.751194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:28.751194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:28.751194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:28.751194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 261/507] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:28.751865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185bc02-efd3-4b89-95be-6dd407d6785c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.793070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 247/508] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:28.795026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c4b26-a4f0-4404-9115-529f6ec01d60 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.799997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:28.857045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185bc02-efd3-4b89-95be-6dd407d6785c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:28.859673 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:28.890578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246c20d-3a32-4ed5-a340-529559a68df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:28.891187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246c20d-3a32-4ed5-a340-529559a68df7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.891461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246c20d-3a32-4ed5-a340-529559a68df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.891732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246c20d-3a32-4ed5-a340-529559a68df7 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.892386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246c20d-3a32-4ed5-a340-529559a68df7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:28.892690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246c20d-3a32-4ed5-a340-529559a68df7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:28.894062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246c20d-3a32-4ed5-a340-529559a68df7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:28.898219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c4b26-a4f0-4404-9115-529f6ec01d60 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:28.908052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 262/509] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:28.908052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246c20d-3a32-4ed5-a340-529559a68df7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:28.935318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68266563-6382-461f-890e-6433dc628bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:28.935864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68266563-6382-461f-890e-6433dc628bea None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:28.936140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68266563-6382-461f-890e-6433dc628bea None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:28.936403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68266563-6382-461f-890e-6433dc628bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:28.946191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68266563-6382-461f-890e-6433dc628bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:28.947638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68266563-6382-461f-890e-6433dc628bea None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-68266563-6382-461f-890e-6433dc628bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:28.961823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:28.962902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:28.963751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:28.963751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:28.963751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:28.963751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 248/510] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:28.963958 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68266563-6382-461f-890e-6433dc628bea None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.010334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246c20d-3a32-4ed5-a340-529559a68df7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.018829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.042893 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ff73c-de0c-43c0-bb45-8dbcbf025f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.043496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ff73c-de0c-43c0-bb45-8dbcbf025f88 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.044400 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ff73c-de0c-43c0-bb45-8dbcbf025f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.044674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ff73c-de0c-43c0-bb45-8dbcbf025f88 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.045305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ff73c-de0c-43c0-bb45-8dbcbf025f88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:29.045610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ff73c-de0c-43c0-bb45-8dbcbf025f88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.046993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ff73c-de0c-43c0-bb45-8dbcbf025f88 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.058459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 263/511] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:29.059250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ff73c-de0c-43c0-bb45-8dbcbf025f88 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.066820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68266563-6382-461f-890e-6433dc628bea None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.087289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.100197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42029-d4f9-4730-8f20-249759a3ceba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.100811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42029-d4f9-4730-8f20-249759a3ceba None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.101092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42029-d4f9-4730-8f20-249759a3ceba None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.101355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42029-d4f9-4730-8f20-249759a3ceba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.112151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b42029-d4f9-4730-8f20-249759a3ceba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.113859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b42029-d4f9-4730-8f20-249759a3ceba None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-28b42029-d4f9-4730-8f20-249759a3ceba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:29.127562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:29.128625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:29.129657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:29.129657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:29.129657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:29.129657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 249/512] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:29.129927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42029-d4f9-4730-8f20-249759a3ceba None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.163152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ff73c-de0c-43c0-bb45-8dbcbf025f88 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.196455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092ad4f-afe7-4990-98df-24cabc6ed2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.197016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092ad4f-afe7-4990-98df-24cabc6ed2dd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.197300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092ad4f-afe7-4990-98df-24cabc6ed2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.197572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092ad4f-afe7-4990-98df-24cabc6ed2dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.207852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092ad4f-afe7-4990-98df-24cabc6ed2dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.209315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092ad4f-afe7-4990-98df-24cabc6ed2dd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c092ad4f-afe7-4990-98df-24cabc6ed2dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:29.224103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:29.225057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:29.225896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:29.225896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:29.225896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:29.225896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 264/513] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:29.225896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092ad4f-afe7-4990-98df-24cabc6ed2dd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.229697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42029-d4f9-4730-8f20-249759a3ceba None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.237276 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.264200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabe903-8f01-493a-8b19-fb1e183fc61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.264825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabe903-8f01-493a-8b19-fb1e183fc61f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.265107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabe903-8f01-493a-8b19-fb1e183fc61f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.265382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabe903-8f01-493a-8b19-fb1e183fc61f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.275642 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabe903-8f01-493a-8b19-fb1e183fc61f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.277099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabe903-8f01-493a-8b19-fb1e183fc61f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-ecabe903-8f01-493a-8b19-fb1e183fc61f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:29.291347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:29.292310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:29.293142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:29.293142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:29.293142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:29.293142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 250/514] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:29.293142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabe903-8f01-493a-8b19-fb1e183fc61f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.309347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.330909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092ad4f-afe7-4990-98df-24cabc6ed2dd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.363362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b758d6-fa1c-4131-a479-9b88f1b4d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.364037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b758d6-fa1c-4131-a479-9b88f1b4d2fc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.364319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b758d6-fa1c-4131-a479-9b88f1b4d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.364591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b758d6-fa1c-4131-a479-9b88f1b4d2fc None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.365217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b758d6-fa1c-4131-a479-9b88f1b4d2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:29.365515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b758d6-fa1c-4131-a479-9b88f1b4d2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.366901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b758d6-fa1c-4131-a479-9b88f1b4d2fc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.378238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 265/515] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:29.379011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b758d6-fa1c-4131-a479-9b88f1b4d2fc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.393040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabe903-8f01-493a-8b19-fb1e183fc61f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.426345 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362cc41-0a59-4be4-931a-b042fb404de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.426972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362cc41-0a59-4be4-931a-b042fb404de3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.427254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362cc41-0a59-4be4-931a-b042fb404de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.427528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362cc41-0a59-4be4-931a-b042fb404de3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.434050 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362cc41-0a59-4be4-931a-b042fb404de3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.435508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362cc41-0a59-4be4-931a-b042fb404de3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-8362cc41-0a59-4be4-931a-b042fb404de3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:29.447099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:29.447929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:29.448752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:29.448752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:29.448752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:29.448752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:29.448752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:29.448752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:29.448752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:29.448752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:29.448752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 251/516] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:29.449219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362cc41-0a59-4be4-931a-b042fb404de3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.469717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.482537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b758d6-fa1c-4131-a479-9b88f1b4d2fc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.515868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652c340-baf6-4d4c-9298-adfcacc05168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.516496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652c340-baf6-4d4c-9298-adfcacc05168 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.516785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652c340-baf6-4d4c-9298-adfcacc05168 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.517058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652c340-baf6-4d4c-9298-adfcacc05168 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.517684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652c340-baf6-4d4c-9298-adfcacc05168 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:29.518022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652c340-baf6-4d4c-9298-adfcacc05168 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.519382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652c340-baf6-4d4c-9298-adfcacc05168 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.525550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.532046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 266/517] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:29.532829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652c340-baf6-4d4c-9298-adfcacc05168 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.553862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362cc41-0a59-4be4-931a-b042fb404de3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.586400 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ab69b-1de6-4b54-b506-d2d6e872b210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.587004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ab69b-1de6-4b54-b506-d2d6e872b210 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.587280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ab69b-1de6-4b54-b506-d2d6e872b210 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.587548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ab69b-1de6-4b54-b506-d2d6e872b210 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.588178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ab69b-1de6-4b54-b506-d2d6e872b210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:29.588506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ab69b-1de6-4b54-b506-d2d6e872b210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.589839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ab69b-1de6-4b54-b506-d2d6e872b210 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.602029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 252/518] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:29.602853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ab69b-1de6-4b54-b506-d2d6e872b210 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.635766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652c340-baf6-4d4c-9298-adfcacc05168 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.669698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fd78e-e171-4089-80b6-87143af8111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.670330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd78e-e171-4089-80b6-87143af8111a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.670615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd78e-e171-4089-80b6-87143af8111a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.670890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd78e-e171-4089-80b6-87143af8111a None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.671551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fd78e-e171-4089-80b6-87143af8111a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:29.671855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fd78e-e171-4089-80b6-87143af8111a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.673205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fd78e-e171-4089-80b6-87143af8111a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.679088 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.685174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 267/519] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:29.686240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fd78e-e171-4089-80b6-87143af8111a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.702516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ab69b-1de6-4b54-b506-d2d6e872b210 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.736701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376de7f0-7a8e-4c49-ba2f-072479cb373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.737318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376de7f0-7a8e-4c49-ba2f-072479cb373b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.737594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376de7f0-7a8e-4c49-ba2f-072479cb373b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.738143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376de7f0-7a8e-4c49-ba2f-072479cb373b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.745287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376de7f0-7a8e-4c49-ba2f-072479cb373b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.747266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376de7f0-7a8e-4c49-ba2f-072479cb373b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.750826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-376de7f0-7a8e-4c49-ba2f-072479cb373b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:29.759918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:29.760802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 05:14:29.761679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:29.762199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 253/520] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:29.764285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376de7f0-7a8e-4c49-ba2f-072479cb373b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.791859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fd78e-e171-4089-80b6-87143af8111a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.824786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8271ff1-f69e-4672-991c-7a7b37742476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.825956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8271ff1-f69e-4672-991c-7a7b37742476 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.826253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8271ff1-f69e-4672-991c-7a7b37742476 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.826529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8271ff1-f69e-4672-991c-7a7b37742476 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.827213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8271ff1-f69e-4672-991c-7a7b37742476 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:29.827520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8271ff1-f69e-4672-991c-7a7b37742476 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.828946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8271ff1-f69e-4672-991c-7a7b37742476 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.846060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 268/521] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:29.846821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8271ff1-f69e-4672-991c-7a7b37742476 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.863614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376de7f0-7a8e-4c49-ba2f-072479cb373b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.897842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9919d-0cfb-4c97-ae27-38023e6feede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.898498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9919d-0cfb-4c97-ae27-38023e6feede None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.898773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9919d-0cfb-4c97-ae27-38023e6feede None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.899043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9919d-0cfb-4c97-ae27-38023e6feede None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.905437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9919d-0cfb-4c97-ae27-38023e6feede None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.906925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9919d-0cfb-4c97-ae27-38023e6feede None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:29.908709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c6d9919d-0cfb-4c97-ae27-38023e6feede None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:29.919110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:29.920143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:29.920975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:29.920975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:29.920975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:29.920975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:29.920975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:29.920975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:29.920975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:29.920975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:29.920975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 254/522] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:29.921431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9919d-0cfb-4c97-ae27-38023e6feede None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:29.947591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8271ff1-f69e-4672-991c-7a7b37742476 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:29.966247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:29.985167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356529da-563e-4460-a1a7-14ab62e4516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:29.985765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356529da-563e-4460-a1a7-14ab62e4516a None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:29.986072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356529da-563e-4460-a1a7-14ab62e4516a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:29.986348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356529da-563e-4460-a1a7-14ab62e4516a None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:29.986986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356529da-563e-4460-a1a7-14ab62e4516a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:29.988650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356529da-563e-4460-a1a7-14ab62e4516a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:30.009620 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 269/523] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:14:29 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:14:30.010448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356529da-563e-4460-a1a7-14ab62e4516a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:30.041411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9919d-0cfb-4c97-ae27-38023e6feede None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:30.080977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c58252-7ee7-43ca-b792-ff644f04d989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:30.081624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c58252-7ee7-43ca-b792-ff644f04d989 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:30.081992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c58252-7ee7-43ca-b792-ff644f04d989 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:30.082274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c58252-7ee7-43ca-b792-ff644f04d989 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:30.091243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c58252-7ee7-43ca-b792-ff644f04d989 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:30.092710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c58252-7ee7-43ca-b792-ff644f04d989 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-a9c58252-7ee7-43ca-b792-ff644f04d989 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:30.105328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:30.107091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:30.107979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 255/524] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:30.108539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c58252-7ee7-43ca-b792-ff644f04d989 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:30.130854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356529da-563e-4460-a1a7-14ab62e4516a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:30.171139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2d9f2-eb4f-4c73-b829-e1eaee0b7c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:30.171852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2d9f2-eb4f-4c73-b829-e1eaee0b7c2d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:30.172014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2d9f2-eb4f-4c73-b829-e1eaee0b7c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:30.172280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2d9f2-eb4f-4c73-b829-e1eaee0b7c2d None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:30.172377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:30.173009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2d9f2-eb4f-4c73-b829-e1eaee0b7c2d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:30.173431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2d9f2-eb4f-4c73-b829-e1eaee0b7c2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:30.174989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2d9f2-eb4f-4c73-b829-e1eaee0b7c2d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:30.215951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 270/525] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Sep 21 05:14:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:30.216750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2d9f2-eb4f-4c73-b829-e1eaee0b7c2d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:30.228022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c58252-7ee7-43ca-b792-ff644f04d989 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:30.252169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:30.263409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6162118-76bd-4a0c-a726-ffe556f50543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:30.264018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6162118-76bd-4a0c-a726-ffe556f50543 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:30.264297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6162118-76bd-4a0c-a726-ffe556f50543 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:30.264809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6162118-76bd-4a0c-a726-ffe556f50543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:30.265347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6162118-76bd-4a0c-a726-ffe556f50543 None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d9168d8bbf8b43299a7748c9faa099b1) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:30.265583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6162118-76bd-4a0c-a726-ffe556f50543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:30.267300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6162118-76bd-4a0c-a726-ffe556f50543 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:30.272397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 256/526] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:30 2022] GET /identity/v3/role_assignments?user.id=d9168d8bbf8b43299a7748c9faa099b1&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:30.278092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6162118-76bd-4a0c-a726-ffe556f50543 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:30.364642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2d9f2-eb4f-4c73-b829-e1eaee0b7c2d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:30.392141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6162118-76bd-4a0c-a726-ffe556f50543 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:30.416174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1484e5b-f941-4481-a5a6-d6cf8df2ebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:30.417402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1484e5b-f941-4481-a5a6-d6cf8df2ebf8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:30.417924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1484e5b-f941-4481-a5a6-d6cf8df2ebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:30.419086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:30.419590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1484e5b-f941-4481-a5a6-d6cf8df2ebf8 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:30.422016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1484e5b-f941-4481-a5a6-d6cf8df2ebf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:30.422658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1484e5b-f941-4481-a5a6-d6cf8df2ebf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:30.426283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1484e5b-f941-4481-a5a6-d6cf8df2ebf8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:30.433153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7244d4-8f75-47ea-89b2-d2d98bedd4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:30.433851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7244d4-8f75-47ea-89b2-d2d98bedd4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:30.434802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7244d4-8f75-47ea-89b2-d2d98bedd4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:30.435344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7244d4-8f75-47ea-89b2-d2d98bedd4a9 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:30.436646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7244d4-8f75-47ea-89b2-d2d98bedd4a9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:30.437649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7244d4-8f75-47ea-89b2-d2d98bedd4a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:30.438869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7244d4-8f75-47ea-89b2-d2d98bedd4a9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:30.444287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 271/527] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:30.445404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1484e5b-f941-4481-a5a6-d6cf8df2ebf8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:30.460251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 257/528] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Sep 21 05:14:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:30.462101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7244d4-8f75-47ea-89b2-d2d98bedd4a9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:30.554981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:30.567811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1484e5b-f941-4481-a5a6-d6cf8df2ebf8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:30.593544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7244d4-8f75-47ea-89b2-d2d98bedd4a9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:30.614426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f533f9-3510-4f0a-8cff-c9b760338fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:30.615202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f533f9-3510-4f0a-8cff-c9b760338fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:30.615547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f533f9-3510-4f0a-8cff-c9b760338fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:30.615918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f533f9-3510-4f0a-8cff-c9b760338fb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:30.631750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f275b-b843-4a8a-be42-ee947244bf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:30.632646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f275b-b843-4a8a-be42-ee947244bf1c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:30.633102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f275b-b843-4a8a-be42-ee947244bf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:30.633581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f275b-b843-4a8a-be42-ee947244bf1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:30.634508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f275b-b843-4a8a-be42-ee947244bf1c None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=5305f8a5d2b540de91d3caedad454bfa) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:30.635041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f275b-b843-4a8a-be42-ee947244bf1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:30.636669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f275b-b843-4a8a-be42-ee947244bf1c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:30.642901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 258/529] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:30 2022] GET /identity/v3/role_assignments?user.id=5305f8a5d2b540de91d3caedad454bfa&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:30.644021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f275b-b843-4a8a-be42-ee947244bf1c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:30.700842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f533f9-3510-4f0a-8cff-c9b760338fb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:30.702391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f533f9-3510-4f0a-8cff-c9b760338fb2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:30.747942 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:30.816462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:30.837790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 272/530] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:30.843572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f533f9-3510-4f0a-8cff-c9b760338fb2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:30.843928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f275b-b843-4a8a-be42-ee947244bf1c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:30.906188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c87d7-f36f-4fe3-83cd-ec212c0e3897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:30.906802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c87d7-f36f-4fe3-83cd-ec212c0e3897 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:30.907079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c87d7-f36f-4fe3-83cd-ec212c0e3897 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:30.907349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c87d7-f36f-4fe3-83cd-ec212c0e3897 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:31.023995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c87d7-f36f-4fe3-83cd-ec212c0e3897 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:31.026304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c87d7-f36f-4fe3-83cd-ec212c0e3897 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:31.029925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f533f9-3510-4f0a-8cff-c9b760338fb2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:31.082330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:31.123243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835996f-e178-41bf-97e5-e4d7093a67dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:31.123948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835996f-e178-41bf-97e5-e4d7093a67dd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:31.124234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835996f-e178-41bf-97e5-e4d7093a67dd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:31.124509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835996f-e178-41bf-97e5-e4d7093a67dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:31.143035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835996f-e178-41bf-97e5-e4d7093a67dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:31.144721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835996f-e178-41bf-97e5-e4d7093a67dd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:31.152743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 259/531] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:31.159339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c87d7-f36f-4fe3-83cd-ec212c0e3897 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-8835996f-e178-41bf-97e5-e4d7093a67dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:31.171510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:31.172786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:31.173632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:31.173632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:31.173632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:31.173632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 273/532] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:31.178014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835996f-e178-41bf-97e5-e4d7093a67dd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:31.252755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:31.315794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835996f-e178-41bf-97e5-e4d7093a67dd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:31.356307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c87d7-f36f-4fe3-83cd-ec212c0e3897 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:31.365856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07bd62-f38e-46fc-9beb-f48e621ef6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:31.366743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07bd62-f38e-46fc-9beb-f48e621ef6cc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:31.367252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07bd62-f38e-46fc-9beb-f48e621ef6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:31.367732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07bd62-f38e-46fc-9beb-f48e621ef6cc None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:31.368574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07bd62-f38e-46fc-9beb-f48e621ef6cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:31.369077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07bd62-f38e-46fc-9beb-f48e621ef6cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:31.370848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07bd62-f38e-46fc-9beb-f48e621ef6cc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:31.385896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 274/533] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:31.394064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07bd62-f38e-46fc-9beb-f48e621ef6cc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:31.430047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33070bd5-ba83-4a02-be8c-1743e35f3ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:31.430680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33070bd5-ba83-4a02-be8c-1743e35f3ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:31.430956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33070bd5-ba83-4a02-be8c-1743e35f3ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:31.431228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33070bd5-ba83-4a02-be8c-1743e35f3ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:31.449109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33070bd5-ba83-4a02-be8c-1743e35f3ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:31.451299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33070bd5-ba83-4a02-be8c-1743e35f3ca8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:31.458811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-33070bd5-ba83-4a02-be8c-1743e35f3ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:31.482410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:31.483595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:31.484442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:31.484442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:31.484442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:31.484728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 260/534] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:31.491057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33070bd5-ba83-4a02-be8c-1743e35f3ca8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:31.606950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33070bd5-ba83-4a02-be8c-1743e35f3ca8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:31.643618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d0d3e-91ae-4c07-b97d-612bc4f757b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:31.644226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d0d3e-91ae-4c07-b97d-612bc4f757b5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:31.644383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07bd62-f38e-46fc-9beb-f48e621ef6cc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:31.644490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d0d3e-91ae-4c07-b97d-612bc4f757b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:31.644762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d0d3e-91ae-4c07-b97d-612bc4f757b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:31.645403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d0d3e-91ae-4c07-b97d-612bc4f757b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:31.645705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d0d3e-91ae-4c07-b97d-612bc4f757b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:31.647121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d0d3e-91ae-4c07-b97d-612bc4f757b5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:31.649792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:31.659690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 261/535] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:31.670932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d0d3e-91ae-4c07-b97d-612bc4f757b5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:31.705894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543fcdb-ffce-45ad-af21-4a0dc219ae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:31.706548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543fcdb-ffce-45ad-af21-4a0dc219ae4a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:31.706825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543fcdb-ffce-45ad-af21-4a0dc219ae4a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:31.707098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543fcdb-ffce-45ad-af21-4a0dc219ae4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:31.717080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543fcdb-ffce-45ad-af21-4a0dc219ae4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:31.718556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543fcdb-ffce-45ad-af21-4a0dc219ae4a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3543fcdb-ffce-45ad-af21-4a0dc219ae4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:31.735929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:31.736767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:31.737846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:31.737846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:31.737846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:31.737846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:31.737846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:31.737846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:31.737846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:31.737846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:31.747823 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 275/536] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:31.750684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543fcdb-ffce-45ad-af21-4a0dc219ae4a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:31.786313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:31.847758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d0d3e-91ae-4c07-b97d-612bc4f757b5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:31.909270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7e3b9-d1d0-49e1-bd59-56eaa104ad5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:31.909816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7e3b9-d1d0-49e1-bd59-56eaa104ad5b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:31.910124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7e3b9-d1d0-49e1-bd59-56eaa104ad5b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:31.910393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7e3b9-d1d0-49e1-bd59-56eaa104ad5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:31.921041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7e3b9-d1d0-49e1-bd59-56eaa104ad5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:31.922548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7e3b9-d1d0-49e1-bd59-56eaa104ad5b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:31.927422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543fcdb-ffce-45ad-af21-4a0dc219ae4a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-faf7e3b9-d1d0-49e1-bd59-56eaa104ad5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:31.944271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:31.945147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:31.945995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:31.945995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:31.945995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:31.945995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:31.945995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:31.945995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:31.945995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:31.945995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:31.947707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 262/537] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:31.952115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7e3b9-d1d0-49e1-bd59-56eaa104ad5b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:32.005389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c0599-d2ae-494c-b539-c9051e0962dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:32.006028 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c0599-d2ae-494c-b539-c9051e0962dc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:32.008387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c0599-d2ae-494c-b539-c9051e0962dc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:32.009306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c0599-d2ae-494c-b539-c9051e0962dc None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:32.010024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c0599-d2ae-494c-b539-c9051e0962dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:32.010335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c0599-d2ae-494c-b539-c9051e0962dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:32.011792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c0599-d2ae-494c-b539-c9051e0962dc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:32.041889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 276/538] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:32.043956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c0599-d2ae-494c-b539-c9051e0962dc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:32.071763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:32.147063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7e3b9-d1d0-49e1-bd59-56eaa104ad5b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:32.151469 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c0599-d2ae-494c-b539-c9051e0962dc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:32.186835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5073e6-2fc3-46e6-9ed7-8277eedb3322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:32.187530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:32.188127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5073e6-2fc3-46e6-9ed7-8277eedb3322 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:32.188302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5073e6-2fc3-46e6-9ed7-8277eedb3322 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:32.190197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5073e6-2fc3-46e6-9ed7-8277eedb3322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:32.190197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5073e6-2fc3-46e6-9ed7-8277eedb3322 None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=c07403facfaf4c71ab5324af6c44f52a) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:32.190197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5073e6-2fc3-46e6-9ed7-8277eedb3322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:32.192646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5073e6-2fc3-46e6-9ed7-8277eedb3322 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:32.200940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 277/539] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:32 2022] GET /identity/v3/role_assignments?user.id=c07403facfaf4c71ab5324af6c44f52a&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:32.242842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07911d-6d59-4785-86fc-8175cf57e871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:32.243481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07911d-6d59-4785-86fc-8175cf57e871 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:32.243760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07911d-6d59-4785-86fc-8175cf57e871 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:32.244029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07911d-6d59-4785-86fc-8175cf57e871 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:32.244746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07911d-6d59-4785-86fc-8175cf57e871 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:32.245051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07911d-6d59-4785-86fc-8175cf57e871 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:32.246557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07911d-6d59-4785-86fc-8175cf57e871 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:32.274437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 263/540] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:32.280149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5073e6-2fc3-46e6-9ed7-8277eedb3322 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:32.414575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:32.428855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5073e6-2fc3-46e6-9ed7-8277eedb3322 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:32.471877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81422d0e-f210-44a8-bed9-65f048b22043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:32.472410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81422d0e-f210-44a8-bed9-65f048b22043 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:32.472685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81422d0e-f210-44a8-bed9-65f048b22043 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:32.472956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81422d0e-f210-44a8-bed9-65f048b22043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:32.473675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81422d0e-f210-44a8-bed9-65f048b22043 None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=63a04998aafd40cc941283700898c92f) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:32.473997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81422d0e-f210-44a8-bed9-65f048b22043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:32.475419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81422d0e-f210-44a8-bed9-65f048b22043 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:32.482135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34216b2-9f9c-4726-baa1-bf138abb91ab None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:32.482707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34216b2-9f9c-4726-baa1-bf138abb91ab None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:32.482707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34216b2-9f9c-4726-baa1-bf138abb91ab None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:32.484259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 264/541] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:32.484821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 278/542] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:32 2022] GET /identity/v3/role_assignments?user.id=63a04998aafd40cc941283700898c92f&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:32.489974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa8170-0ea3-4c9c-8f60-0786ea107678 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:32.490269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa8170-0ea3-4c9c-8f60-0786ea107678 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:32.490543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa8170-0ea3-4c9c-8f60-0786ea107678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:32.521325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ffaa8170-0ea3-4c9c-8f60-0786ea107678 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:32.522572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ffaa8170-0ea3-4c9c-8f60-0786ea107678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:32.662691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 265/543] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:33.006538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bb2ed-1dca-446a-9401-018a9b58febf None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.007234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bb2ed-1dca-446a-9401-018a9b58febf None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.007710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bb2ed-1dca-446a-9401-018a9b58febf None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:33.009464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 279/544] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:33.014930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa6668-5c4f-4913-bb3a-a94f414073e6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.015219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa6668-5c4f-4913-bb3a-a94f414073e6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.015500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa6668-5c4f-4913-bb3a-a94f414073e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:33.038964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c1fa6668-5c4f-4913-bb3a-a94f414073e6 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:33.040477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c1fa6668-5c4f-4913-bb3a-a94f414073e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:33.071520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bb2ed-1dca-446a-9401-018a9b58febf None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:33.138002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 266/545] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:33.203166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883169cc-6051-4506-93fd-ab7522f5db02 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.203460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883169cc-6051-4506-93fd-ab7522f5db02 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.203720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883169cc-6051-4506-93fd-ab7522f5db02 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:33.205415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 267/546] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:33.223126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980236b-de98-4618-b7c6-ce776214b144 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.223401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980236b-de98-4618-b7c6-ce776214b144 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.223666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980236b-de98-4618-b7c6-ce776214b144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:33.241298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bb2ed-1dca-446a-9401-018a9b58febf None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:33.259022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2980236b-de98-4618-b7c6-ce776214b144 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:33.260152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2980236b-de98-4618-b7c6-ce776214b144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:33.275671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2094b2-7170-4d20-bc85-990deb94f055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:33.276306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2094b2-7170-4d20-bc85-990deb94f055 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.276586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2094b2-7170-4d20-bc85-990deb94f055 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.276868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2094b2-7170-4d20-bc85-990deb94f055 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:33.303045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2094b2-7170-4d20-bc85-990deb94f055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:33.308327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5d2094b2-7170-4d20-bc85-990deb94f055 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:33.309396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:33.318619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:33.319438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:33.323385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 280/547] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:33.328748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2094b2-7170-4d20-bc85-990deb94f055 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:33.354559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:33.401154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 268/548] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:33.459547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980236b-de98-4618-b7c6-ce776214b144 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:33.523988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2094b2-7170-4d20-bc85-990deb94f055 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:33.589097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95ec17-a5f2-4656-bd8a-7fe678f36c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:33.590104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95ec17-a5f2-4656-bd8a-7fe678f36c81 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.590647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95ec17-a5f2-4656-bd8a-7fe678f36c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.591122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95ec17-a5f2-4656-bd8a-7fe678f36c81 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:33.593838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95ec17-a5f2-4656-bd8a-7fe678f36c81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:33.594178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95ec17-a5f2-4656-bd8a-7fe678f36c81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:33.595546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95ec17-a5f2-4656-bd8a-7fe678f36c81 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:33.608676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 281/549] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:33.614848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95ec17-a5f2-4656-bd8a-7fe678f36c81 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:33.639407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980236b-de98-4618-b7c6-ce776214b144 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:33.692903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593270b-85c2-455f-a5d0-a8f0263e69d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:33.694186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593270b-85c2-455f-a5d0-a8f0263e69d4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.694687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593270b-85c2-455f-a5d0-a8f0263e69d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.695147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593270b-85c2-455f-a5d0-a8f0263e69d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:33.707172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593270b-85c2-455f-a5d0-a8f0263e69d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0593270b-85c2-455f-a5d0-a8f0263e69d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:33.712864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:33.713727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:33.714624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:33.717804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 269/550] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:33.723692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593270b-85c2-455f-a5d0-a8f0263e69d4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:33.753449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:33.779771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95ec17-a5f2-4656-bd8a-7fe678f36c81 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:33.786296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:33.816490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c346e7-69e9-45d2-91ea-7ca3958fed20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:33.817265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c346e7-69e9-45d2-91ea-7ca3958fed20 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.817686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c346e7-69e9-45d2-91ea-7ca3958fed20 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.818101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c346e7-69e9-45d2-91ea-7ca3958fed20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:33.829612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c346e7-69e9-45d2-91ea-7ca3958fed20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:33.831279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c346e7-69e9-45d2-91ea-7ca3958fed20 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-60c346e7-69e9-45d2-91ea-7ca3958fed20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:33.845652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:33.846935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:33.847773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:33.847773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:33.847773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:33.847944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 282/551] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:33.849007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c346e7-69e9-45d2-91ea-7ca3958fed20 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:33.861615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593270b-85c2-455f-a5d0-a8f0263e69d4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:33.899695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de8725-fe6a-4a21-a219-ab5fdb83b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:33.900727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de8725-fe6a-4a21-a219-ab5fdb83b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.901331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de8725-fe6a-4a21-a219-ab5fdb83b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.901805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de8725-fe6a-4a21-a219-ab5fdb83b0d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:33.902684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de8725-fe6a-4a21-a219-ab5fdb83b0d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:33.903213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de8725-fe6a-4a21-a219-ab5fdb83b0d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:33.904886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de8725-fe6a-4a21-a219-ab5fdb83b0d9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:33.913838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 270/552] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:33.914892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de8725-fe6a-4a21-a219-ab5fdb83b0d9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:33.957240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c346e7-69e9-45d2-91ea-7ca3958fed20 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:33.984568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:33.994591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f01e91-b720-4b12-86ea-83b626453b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:33.995297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f01e91-b720-4b12-86ea-83b626453b82 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:33.995626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f01e91-b720-4b12-86ea-83b626453b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:33.995897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f01e91-b720-4b12-86ea-83b626453b82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.007206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f01e91-b720-4b12-86ea-83b626453b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f3f01e91-b720-4b12-86ea-83b626453b82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:34.012545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:34.013640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:34.014501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:34.018575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 283/553] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:34.019521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f01e91-b720-4b12-86ea-83b626453b82 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:34.020986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:34.041064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de8725-fe6a-4a21-a219-ab5fdb83b0d9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.077699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f141f-f10a-442b-85c9-2840cbeaf1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:34.078341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f141f-f10a-442b-85c9-2840cbeaf1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.078617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f141f-f10a-442b-85c9-2840cbeaf1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.078885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f141f-f10a-442b-85c9-2840cbeaf1e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.079693 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f141f-f10a-442b-85c9-2840cbeaf1e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:34.079799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f141f-f10a-442b-85c9-2840cbeaf1e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:34.081161 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f141f-f10a-442b-85c9-2840cbeaf1e7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:34.093253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 271/554] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:34.098489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f141f-f10a-442b-85c9-2840cbeaf1e7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:34.151601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f01e91-b720-4b12-86ea-83b626453b82 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.188085 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdc463-968c-4768-b461-e6643b2e6e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:34.188718 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdc463-968c-4768-b461-e6643b2e6e5c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.188997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdc463-968c-4768-b461-e6643b2e6e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.189443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdc463-968c-4768-b461-e6643b2e6e5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.201511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bdc463-968c-4768-b461-e6643b2e6e5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:34.203145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bdc463-968c-4768-b461-e6643b2e6e5c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:34.216211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f141f-f10a-442b-85c9-2840cbeaf1e7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-24bdc463-968c-4768-b461-e6643b2e6e5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:34.219396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:34.220355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:34.221206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:34.221206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:34.221206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:34.224471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 284/555] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:34.225297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdc463-968c-4768-b461-e6643b2e6e5c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:34.256645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f49522-e117-463c-9d25-f08d8850d2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:34.257625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f49522-e117-463c-9d25-f08d8850d2ed None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.258177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f49522-e117-463c-9d25-f08d8850d2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.258621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f49522-e117-463c-9d25-f08d8850d2ed None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.259515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f49522-e117-463c-9d25-f08d8850d2ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:34.259986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f49522-e117-463c-9d25-f08d8850d2ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:34.261615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f49522-e117-463c-9d25-f08d8850d2ed None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:34.266904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:34.276149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 272/556] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:34.278677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f49522-e117-463c-9d25-f08d8850d2ed None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:34.292888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:34.335378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdc463-968c-4768-b461-e6643b2e6e5c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.399478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f49522-e117-463c-9d25-f08d8850d2ed None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.404673 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070210c-9ba1-423d-8e69-3d06c531695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:34.405319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070210c-9ba1-423d-8e69-3d06c531695d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.405597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070210c-9ba1-423d-8e69-3d06c531695d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.405864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070210c-9ba1-423d-8e69-3d06c531695d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.417833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070210c-9ba1-423d-8e69-3d06c531695d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:34.420058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070210c-9ba1-423d-8e69-3d06c531695d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-e070210c-9ba1-423d-8e69-3d06c531695d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:34.438090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:34.439557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:14:34.440476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:34.441596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e919b6-f376-4482-9320-96e6ff65434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:34.442412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e919b6-f376-4482-9320-96e6ff65434d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.442870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e919b6-f376-4482-9320-96e6ff65434d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.443293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e919b6-f376-4482-9320-96e6ff65434d None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.444086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e919b6-f376-4482-9320-96e6ff65434d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:34.444541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e919b6-f376-4482-9320-96e6ff65434d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:34.446080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e919b6-f376-4482-9320-96e6ff65434d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:34.449175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 285/557] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:34.450145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070210c-9ba1-423d-8e69-3d06c531695d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:34.461018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 273/558] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:34.462180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e919b6-f376-4482-9320-96e6ff65434d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:34.542669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:34.582562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e919b6-f376-4482-9320-96e6ff65434d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.599173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:34.603158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070210c-9ba1-423d-8e69-3d06c531695d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.625519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996a1b-a633-46e0-a3dd-c9970fe3bbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:34.626690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996a1b-a633-46e0-a3dd-c9970fe3bbee None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.627274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996a1b-a633-46e0-a3dd-c9970fe3bbee None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.627765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996a1b-a633-46e0-a3dd-c9970fe3bbee None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.628682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996a1b-a633-46e0-a3dd-c9970fe3bbee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:34.629180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996a1b-a633-46e0-a3dd-c9970fe3bbee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:34.630833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996a1b-a633-46e0-a3dd-c9970fe3bbee None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:34.655567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 274/559] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Sep 21 05:14:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:34.657275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996a1b-a633-46e0-a3dd-c9970fe3bbee None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:34.675980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1662e51-a670-4069-97a6-677ea4fbeac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:34.676606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1662e51-a670-4069-97a6-677ea4fbeac7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.677084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1662e51-a670-4069-97a6-677ea4fbeac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.677357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1662e51-a670-4069-97a6-677ea4fbeac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.693620 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1662e51-a670-4069-97a6-677ea4fbeac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:34.695249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1662e51-a670-4069-97a6-677ea4fbeac7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-e1662e51-a670-4069-97a6-677ea4fbeac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:34.714852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:34.716482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:34.717334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:34.717334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:34.717334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:34.719291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 286/560] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:34.720437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1662e51-a670-4069-97a6-677ea4fbeac7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:34.781652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996a1b-a633-46e0-a3dd-c9970fe3bbee None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.814440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:34.823170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0faabc8-ce47-45d5-a014-bc1e51322a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:34.823806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0faabc8-ce47-45d5-a014-bc1e51322a62 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.824192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0faabc8-ce47-45d5-a014-bc1e51322a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.824469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0faabc8-ce47-45d5-a014-bc1e51322a62 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.834257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0faabc8-ce47-45d5-a014-bc1e51322a62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:34.836043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0faabc8-ce47-45d5-a014-bc1e51322a62 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:34.839451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1662e51-a670-4069-97a6-677ea4fbeac7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-f0faabc8-ce47-45d5-a014-bc1e51322a62 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:34.849742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:34.850882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 21 05:14:34.851845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:34.856152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 275/561] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:34.857444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c053deb-345c-4d92-8ef6-0c6f8d3422ba None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.857901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c053deb-345c-4d92-8ef6-0c6f8d3422ba None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.858154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c053deb-345c-4d92-8ef6-0c6f8d3422ba None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.860083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 276/562] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:34.860908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c053deb-345c-4d92-8ef6-0c6f8d3422ba None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:34.896455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ab477-bc88-46f6-b753-3c40513e16f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:34.897241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ab477-bc88-46f6-b753-3c40513e16f5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:34.897622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ab477-bc88-46f6-b753-3c40513e16f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:34.898281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ab477-bc88-46f6-b753-3c40513e16f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:34.931583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:34.987029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c053deb-345c-4d92-8ef6-0c6f8d3422ba None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:34.991494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ab477-bc88-46f6-b753-3c40513e16f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:34.993872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ab477-bc88-46f6-b753-3c40513e16f5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.022282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799740c-d92b-4ebc-81ba-502e2234abd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.022883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799740c-d92b-4ebc-81ba-502e2234abd1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.023160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799740c-d92b-4ebc-81ba-502e2234abd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.023429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799740c-d92b-4ebc-81ba-502e2234abd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.024060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799740c-d92b-4ebc-81ba-502e2234abd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:35.024360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799740c-d92b-4ebc-81ba-502e2234abd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:35.025711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799740c-d92b-4ebc-81ba-502e2234abd1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.037041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 277/563] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:35.037815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799740c-d92b-4ebc-81ba-502e2234abd1 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.086537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:35.093771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 287/564] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:35.095478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa8bb1-3312-4a5c-8681-74364efc662c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.095765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa8bb1-3312-4a5c-8681-74364efc662c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.096040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa8bb1-3312-4a5c-8681-74364efc662c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.113922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-54fa8bb1-3312-4a5c-8681-74364efc662c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:35.115132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-54fa8bb1-3312-4a5c-8681-74364efc662c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:35.136717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799740c-d92b-4ebc-81ba-502e2234abd1 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:35.161157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:35.170348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09916d-2d9e-46b2-b467-7ec364a95099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.170943 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09916d-2d9e-46b2-b467-7ec364a95099 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.171223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09916d-2d9e-46b2-b467-7ec364a95099 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.171491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09916d-2d9e-46b2-b467-7ec364a95099 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.172204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09916d-2d9e-46b2-b467-7ec364a95099 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:35.172504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09916d-2d9e-46b2-b467-7ec364a95099 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:35.173913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09916d-2d9e-46b2-b467-7ec364a95099 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.181817 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 288/565] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:35.182890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee569-8d1e-43ad-845d-8e3123ffa7c1 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.183174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee569-8d1e-43ad-845d-8e3123ffa7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.183445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee569-8d1e-43ad-845d-8e3123ffa7c1 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.185189 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 289/566] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:35.185872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee569-8d1e-43ad-845d-8e3123ffa7c1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.202980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 278/567] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:35.203151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09916d-2d9e-46b2-b467-7ec364a95099 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.293969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee569-8d1e-43ad-845d-8e3123ffa7c1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:35.307396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09916d-2d9e-46b2-b467-7ec364a95099 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:35.315822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:35.330807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8876b5-3400-465b-9101-0190d6534910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.331435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8876b5-3400-465b-9101-0190d6534910 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.331716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8876b5-3400-465b-9101-0190d6534910 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.331990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8876b5-3400-465b-9101-0190d6534910 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.339783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8876b5-3400-465b-9101-0190d6534910 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:35.341659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8876b5-3400-465b-9101-0190d6534910 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.343412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f47269-1839-4a5f-b1ce-72e4d7207f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.344033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f47269-1839-4a5f-b1ce-72e4d7207f30 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.344315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f47269-1839-4a5f-b1ce-72e4d7207f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.344594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f47269-1839-4a5f-b1ce-72e4d7207f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.354442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f47269-1839-4a5f-b1ce-72e4d7207f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-8c8876b5-3400-465b-9101-0190d6534910 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:35.354684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:35.355566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:35.357056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:35.357593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f47269-1839-4a5f-b1ce-72e4d7207f30 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.358325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 290/568] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:35.359290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba444fcd-8630-48d3-9fdf-ed7c4695acd1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.359570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba444fcd-8630-48d3-9fdf-ed7c4695acd1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.359841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba444fcd-8630-48d3-9fdf-ed7c4695acd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c0f47269-1839-4a5f-b1ce-72e4d7207f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:35.380549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:35.381520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:35.382432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:35.382432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:35.382432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:35.382432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 279/569] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:35.382432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f47269-1839-4a5f-b1ce-72e4d7207f30 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.384355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ba444fcd-8630-48d3-9fdf-ed7c4695acd1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:35.385521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ba444fcd-8630-48d3-9fdf-ed7c4695acd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:35.423932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:35.459826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 291/570] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:35.460653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba444fcd-8630-48d3-9fdf-ed7c4695acd1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.486962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f47269-1839-4a5f-b1ce-72e4d7207f30 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:35.521585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f7449-5b52-4038-a22c-9a33758f1785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.522243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f7449-5b52-4038-a22c-9a33758f1785 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.522523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f7449-5b52-4038-a22c-9a33758f1785 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.522795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f7449-5b52-4038-a22c-9a33758f1785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.533686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f7449-5b52-4038-a22c-9a33758f1785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:35.535174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f7449-5b52-4038-a22c-9a33758f1785 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5a9f7449-5b52-4038-a22c-9a33758f1785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:35.549672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:35.550649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:35.551519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:35.551519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:35.551519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:35.551519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 280/571] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:35.551751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f7449-5b52-4038-a22c-9a33758f1785 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.564871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:35.577244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba444fcd-8630-48d3-9fdf-ed7c4695acd1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:35.616551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564400a-21bf-461a-9418-8bfe1ba46220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.617192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564400a-21bf-461a-9418-8bfe1ba46220 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.617474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564400a-21bf-461a-9418-8bfe1ba46220 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.618020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564400a-21bf-461a-9418-8bfe1ba46220 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.618503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564400a-21bf-461a-9418-8bfe1ba46220 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:35.618851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564400a-21bf-461a-9418-8bfe1ba46220 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:35.620266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564400a-21bf-461a-9418-8bfe1ba46220 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.639706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 292/572] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:35.641203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564400a-21bf-461a-9418-8bfe1ba46220 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.662168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f7449-5b52-4038-a22c-9a33758f1785 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:35.669948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:35.698735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6621a2d-8eee-44b3-ae9e-a06080a5f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.699699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6621a2d-8eee-44b3-ae9e-a06080a5f935 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.699985 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6621a2d-8eee-44b3-ae9e-a06080a5f935 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.700256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6621a2d-8eee-44b3-ae9e-a06080a5f935 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.701048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6621a2d-8eee-44b3-ae9e-a06080a5f935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:35.701364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6621a2d-8eee-44b3-ae9e-a06080a5f935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:35.703302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6621a2d-8eee-44b3-ae9e-a06080a5f935 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.722940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 281/573] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:35.722940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6621a2d-8eee-44b3-ae9e-a06080a5f935 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.747459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564400a-21bf-461a-9418-8bfe1ba46220 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:35.780881 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff96bc2-6a57-4d4d-96ba-a87baa192736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.781507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff96bc2-6a57-4d4d-96ba-a87baa192736 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.781795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff96bc2-6a57-4d4d-96ba-a87baa192736 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.782111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff96bc2-6a57-4d4d-96ba-a87baa192736 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.791827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:35.792478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff96bc2-6a57-4d4d-96ba-a87baa192736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-9ff96bc2-6a57-4d4d-96ba-a87baa192736 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:35.797575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:35.798716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:35.799560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 293/574] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:35.800244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff96bc2-6a57-4d4d-96ba-a87baa192736 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.824687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6621a2d-8eee-44b3-ae9e-a06080a5f935 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:35.859075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b7fc8-3e41-4f5b-b62a-d56d90c597a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.859608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b7fc8-3e41-4f5b-b62a-d56d90c597a5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.859886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b7fc8-3e41-4f5b-b62a-d56d90c597a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.860159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b7fc8-3e41-4f5b-b62a-d56d90c597a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.860790 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b7fc8-3e41-4f5b-b62a-d56d90c597a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:35.861094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b7fc8-3e41-4f5b-b62a-d56d90c597a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:35.862492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b7fc8-3e41-4f5b-b62a-d56d90c597a5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.873567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 282/575] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:35.874465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b7fc8-3e41-4f5b-b62a-d56d90c597a5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.888230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:35.902427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff96bc2-6a57-4d4d-96ba-a87baa192736 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:35.935984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbd80b-e1e2-4d94-83c6-608df571d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:35.936607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbd80b-e1e2-4d94-83c6-608df571d0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:35.936889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbd80b-e1e2-4d94-83c6-608df571d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:35.937166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbd80b-e1e2-4d94-83c6-608df571d0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:35.947330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbd80b-e1e2-4d94-83c6-608df571d0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:35.948747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbd80b-e1e2-4d94-83c6-608df571d0c1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-37bbd80b-e1e2-4d94-83c6-608df571d0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:35.962847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:35.968898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:35.970123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:35.970123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:35.970123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:35.970123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 294/576] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:35.970123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbd80b-e1e2-4d94-83c6-608df571d0c1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:35.972529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b7fc8-3e41-4f5b-b62a-d56d90c597a5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.000878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:36.007183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee3511-2e06-4a8c-8f7c-b5f3927ee6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.007867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee3511-2e06-4a8c-8f7c-b5f3927ee6ec None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.008107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee3511-2e06-4a8c-8f7c-b5f3927ee6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.008355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee3511-2e06-4a8c-8f7c-b5f3927ee6ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.016085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee3511-2e06-4a8c-8f7c-b5f3927ee6ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.017634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee3511-2e06-4a8c-8f7c-b5f3927ee6ec None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-67ee3511-2e06-4a8c-8f7c-b5f3927ee6ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:36.028611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:36.029519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:36.030831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:36.030831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:36.030831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:36.030831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:36.030831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:36.030831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:36.030831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:36.030831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:36.030831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 283/577] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:36.034729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee3511-2e06-4a8c-8f7c-b5f3927ee6ec None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.077830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbd80b-e1e2-4d94-83c6-608df571d0c1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.112277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac011ce-fe2f-4eea-ba65-1bc0ee6048b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.113129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac011ce-fe2f-4eea-ba65-1bc0ee6048b6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.113406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac011ce-fe2f-4eea-ba65-1bc0ee6048b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.113728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac011ce-fe2f-4eea-ba65-1bc0ee6048b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.121273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:36.125421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac011ce-fe2f-4eea-ba65-1bc0ee6048b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-6ac011ce-fe2f-4eea-ba65-1bc0ee6048b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:36.130413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:36.131378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:36.132202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 295/578] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:36.132895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac011ce-fe2f-4eea-ba65-1bc0ee6048b6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.139879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee3511-2e06-4a8c-8f7c-b5f3927ee6ec None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.173069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b8d03-3550-421e-9558-ccae34f89516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.173664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b8d03-3550-421e-9558-ccae34f89516 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.174020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b8d03-3550-421e-9558-ccae34f89516 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.174309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b8d03-3550-421e-9558-ccae34f89516 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.174962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b8d03-3550-421e-9558-ccae34f89516 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:36.175264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b8d03-3550-421e-9558-ccae34f89516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.176614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b8d03-3550-421e-9558-ccae34f89516 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.184879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 284/579] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:36.185781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b8d03-3550-421e-9558-ccae34f89516 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.230818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac011ce-fe2f-4eea-ba65-1bc0ee6048b6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.237247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:36.265496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac9633-8012-4533-8081-11e85a888d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.266158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac9633-8012-4533-8081-11e85a888d7a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.266439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac9633-8012-4533-8081-11e85a888d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.266710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac9633-8012-4533-8081-11e85a888d7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.273117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac9633-8012-4533-8081-11e85a888d7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.274597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac9633-8012-4533-8081-11e85a888d7a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.285106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b8d03-3550-421e-9558-ccae34f89516 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-89ac9633-8012-4533-8081-11e85a888d7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:36.285556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:36.286407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 296/580] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:36.287491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac9633-8012-4533-8081-11e85a888d7a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.326129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485fcfe-c683-4089-b52d-ff75dc902167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.326853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485fcfe-c683-4089-b52d-ff75dc902167 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.327201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485fcfe-c683-4089-b52d-ff75dc902167 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.327482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485fcfe-c683-4089-b52d-ff75dc902167 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.328118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485fcfe-c683-4089-b52d-ff75dc902167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:36.328422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485fcfe-c683-4089-b52d-ff75dc902167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.329789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485fcfe-c683-4089-b52d-ff75dc902167 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.343040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 285/581] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:36.344055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485fcfe-c683-4089-b52d-ff75dc902167 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.356990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:36.397952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac9633-8012-4533-8081-11e85a888d7a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.430944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649314-64d6-4611-8b88-f22dd9271f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.431546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649314-64d6-4611-8b88-f22dd9271f8d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.431829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649314-64d6-4611-8b88-f22dd9271f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.432106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649314-64d6-4611-8b88-f22dd9271f8d None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.432787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d649314-64d6-4611-8b88-f22dd9271f8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:36.433100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d649314-64d6-4611-8b88-f22dd9271f8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.435193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d649314-64d6-4611-8b88-f22dd9271f8d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.444683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485fcfe-c683-4089-b52d-ff75dc902167 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.446813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 297/582] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:36.447669 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649314-64d6-4611-8b88-f22dd9271f8d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.457498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:36.478969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3de90e-dc5c-4401-9819-0cb5621fbe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.479643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3de90e-dc5c-4401-9819-0cb5621fbe31 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.479986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3de90e-dc5c-4401-9819-0cb5621fbe31 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.480319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3de90e-dc5c-4401-9819-0cb5621fbe31 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.481618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3de90e-dc5c-4401-9819-0cb5621fbe31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:36.481922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3de90e-dc5c-4401-9819-0cb5621fbe31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.483285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3de90e-dc5c-4401-9819-0cb5621fbe31 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.491112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 286/583] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:36.491873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3de90e-dc5c-4401-9819-0cb5621fbe31 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.553899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649314-64d6-4611-8b88-f22dd9271f8d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.573671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:36.589906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4696c-7bec-4216-a6d0-0baaacd979a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.591156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4696c-7bec-4216-a6d0-0baaacd979a2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.591494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4696c-7bec-4216-a6d0-0baaacd979a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.591858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4696c-7bec-4216-a6d0-0baaacd979a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.593592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3de90e-dc5c-4401-9819-0cb5621fbe31 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.602196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4696c-7bec-4216-a6d0-0baaacd979a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.603643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4696c-7bec-4216-a6d0-0baaacd979a2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-98c4696c-7bec-4216-a6d0-0baaacd979a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:36.618605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:36.624452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:36.625609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:36.625609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:36.625609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:36.625609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 298/584] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:36.625609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4696c-7bec-4216-a6d0-0baaacd979a2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.629012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870fc6c-75fb-42ce-b59a-35125ee651bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.629608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870fc6c-75fb-42ce-b59a-35125ee651bd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.629881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870fc6c-75fb-42ce-b59a-35125ee651bd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.630178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870fc6c-75fb-42ce-b59a-35125ee651bd None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.630854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870fc6c-75fb-42ce-b59a-35125ee651bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:36.631156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870fc6c-75fb-42ce-b59a-35125ee651bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.632560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870fc6c-75fb-42ce-b59a-35125ee651bd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.644531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 287/585] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:36.645382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870fc6c-75fb-42ce-b59a-35125ee651bd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.680078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:36.724131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4696c-7bec-4216-a6d0-0baaacd979a2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.743566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870fc6c-75fb-42ce-b59a-35125ee651bd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.757133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806bd4b-5d27-482e-a95f-4601fb4049ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.757755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806bd4b-5d27-482e-a95f-4601fb4049ce None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.758069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806bd4b-5d27-482e-a95f-4601fb4049ce None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.758352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806bd4b-5d27-482e-a95f-4601fb4049ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.764778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806bd4b-5d27-482e-a95f-4601fb4049ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.766248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806bd4b-5d27-482e-a95f-4601fb4049ce None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.776091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02045b1e-08ba-4a50-b435-7d5539219fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.776680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02045b1e-08ba-4a50-b435-7d5539219fdf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.776960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02045b1e-08ba-4a50-b435-7d5539219fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a806bd4b-5d27-482e-a95f-4601fb4049ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:36.777099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:36.778653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:36.779721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:36.779721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:36.779721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:36.779721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:36.779721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:36.779721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:36.779721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:36.779721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:36.780235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02045b1e-08ba-4a50-b435-7d5539219fdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.780235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02045b1e-08ba-4a50-b435-7d5539219fdf None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=ee0fe35d9e304ee495998c93c6909fa8) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:36.780397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 299/586] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:36.780397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806bd4b-5d27-482e-a95f-4601fb4049ce None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.780667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02045b1e-08ba-4a50-b435-7d5539219fdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.782325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02045b1e-08ba-4a50-b435-7d5539219fdf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.788415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 288/587] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/role_assignments?user.id=ee0fe35d9e304ee495998c93c6909fa8&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:36.789590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02045b1e-08ba-4a50-b435-7d5539219fdf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.813527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:36.899012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806bd4b-5d27-482e-a95f-4601fb4049ce None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.902635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:36.907279 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02045b1e-08ba-4a50-b435-7d5539219fdf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:36.937450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18db498-5ee1-4c5d-b3b1-b3355b39e874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.937808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18db498-5ee1-4c5d-b3b1-b3355b39e874 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.938118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18db498-5ee1-4c5d-b3b1-b3355b39e874 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.938415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18db498-5ee1-4c5d-b3b1-b3355b39e874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.947135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc731d0-61e7-4965-adca-0a880f79e9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:36.947745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc731d0-61e7-4965-adca-0a880f79e9ba None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:36.948021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc731d0-61e7-4965-adca-0a880f79e9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:36.948291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc731d0-61e7-4965-adca-0a880f79e9ba None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:36.948938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc731d0-61e7-4965-adca-0a880f79e9ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:36.950192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc731d0-61e7-4965-adca-0a880f79e9ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.951176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc731d0-61e7-4965-adca-0a880f79e9ba None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.954509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18db498-5ee1-4c5d-b3b1-b3355b39e874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:36.956060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18db498-5ee1-4c5d-b3b1-b3355b39e874 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:36.964193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 289/588] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:36.965090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc731d0-61e7-4965-adca-0a880f79e9ba None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b18db498-5ee1-4c5d-b3b1-b3355b39e874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:36.975613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:36.976633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:36.977515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:36.977515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:36.977515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:36.977515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 300/589] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:36.977515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18db498-5ee1-4c5d-b3b1-b3355b39e874 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:37.067263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:37.081830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc731d0-61e7-4965-adca-0a880f79e9ba None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:37.086470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18db498-5ee1-4c5d-b3b1-b3355b39e874 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:37.124979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba3020-fec7-4472-92cb-52426344e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:37.126076 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba3020-fec7-4472-92cb-52426344e2de None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:37.126207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06f967-22ed-4a5c-a6ea-9b57183b8c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:37.126699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba3020-fec7-4472-92cb-52426344e2de None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:37.126804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06f967-22ed-4a5c-a6ea-9b57183b8c24 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:37.127025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06f967-22ed-4a5c-a6ea-9b57183b8c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:37.127297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06f967-22ed-4a5c-a6ea-9b57183b8c24 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/5305f8a5d2b540de91d3caedad454bfa/roles/99269efe3b2a430e9cc7912268d2be84` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:37.127661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba3020-fec7-4472-92cb-52426344e2de None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:37.128586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba3020-fec7-4472-92cb-52426344e2de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:37.128984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba3020-fec7-4472-92cb-52426344e2de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:37.130539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba3020-fec7-4472-92cb-52426344e2de None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:37.148530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 301/590] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:37.148530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba3020-fec7-4472-92cb-52426344e2de None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:37.150679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06f967-22ed-4a5c-a6ea-9b57183b8c24 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=5305f8a5d2b540de91d3caedad454bfa, role_id=99269efe3b2a430e9cc7912268d2be84)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:37.153903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06f967-22ed-4a5c-a6ea-9b57183b8c24 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:37.182038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 290/591] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:14:36 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/5305f8a5d2b540de91d3caedad454bfa/roles/99269efe3b2a430e9cc7912268d2be84 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:14:37.183084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06f967-22ed-4a5c-a6ea-9b57183b8c24 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:37.205374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:37.280604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba3020-fec7-4472-92cb-52426344e2de None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:37.320012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689efa5c-345a-4138-8544-e82d1607ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:37.320657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689efa5c-345a-4138-8544-e82d1607ea7c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:37.320935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689efa5c-345a-4138-8544-e82d1607ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:37.321205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689efa5c-345a-4138-8544-e82d1607ea7c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:37.330548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689efa5c-345a-4138-8544-e82d1607ea7c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:37.332151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689efa5c-345a-4138-8544-e82d1607ea7c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:37.338663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06f967-22ed-4a5c-a6ea-9b57183b8c24 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-689efa5c-345a-4138-8544-e82d1607ea7c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:37.345443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:37.346521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:37.347504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 302/592] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:37.348127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689efa5c-345a-4138-8544-e82d1607ea7c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:37.354269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:37.388035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c38e57-eb43-4ac1-be6d-ddbbaf4679a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:37.388643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c38e57-eb43-4ac1-be6d-ddbbaf4679a9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:37.388923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c38e57-eb43-4ac1-be6d-ddbbaf4679a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:37.389196 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c38e57-eb43-4ac1-be6d-ddbbaf4679a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:37.389831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c38e57-eb43-4ac1-be6d-ddbbaf4679a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:37.390162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c38e57-eb43-4ac1-be6d-ddbbaf4679a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:37.391528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c38e57-eb43-4ac1-be6d-ddbbaf4679a9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:37.406005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 291/593] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:37.407170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c38e57-eb43-4ac1-be6d-ddbbaf4679a9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:37.465182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689efa5c-345a-4138-8544-e82d1607ea7c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:37.484840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:37.503925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400baa4-18b0-42c6-acdb-509ec9bc26f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:37.504586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400baa4-18b0-42c6-acdb-509ec9bc26f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:37.504863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400baa4-18b0-42c6-acdb-509ec9bc26f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:37.505130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400baa4-18b0-42c6-acdb-509ec9bc26f5 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d9168d8bbf8b43299a7748c9faa099b1/roles/99269efe3b2a430e9cc7912268d2be84` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:37.522131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400baa4-18b0-42c6-acdb-509ec9bc26f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=d9168d8bbf8b43299a7748c9faa099b1, role_id=99269efe3b2a430e9cc7912268d2be84)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:37.523599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400baa4-18b0-42c6-acdb-509ec9bc26f5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:37.533187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c38e57-eb43-4ac1-be6d-ddbbaf4679a9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:37.539637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 303/594] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:14:37 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d9168d8bbf8b43299a7748c9faa099b1/roles/99269efe3b2a430e9cc7912268d2be84 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:14:37.546201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400baa4-18b0-42c6-acdb-509ec9bc26f5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:37.626224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e1adb-7535-47c3-826a-c9b1f0316e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:37.626846 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e1adb-7535-47c3-826a-c9b1f0316e32 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:37.627133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e1adb-7535-47c3-826a-c9b1f0316e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:37.627390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e1adb-7535-47c3-826a-c9b1f0316e32 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:37.628136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e1adb-7535-47c3-826a-c9b1f0316e32 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:37.628439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e1adb-7535-47c3-826a-c9b1f0316e32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:37.631549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e1adb-7535-47c3-826a-c9b1f0316e32 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:37.671347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 292/595] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Sep 21 05:14:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:37.676128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:37.677101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e1adb-7535-47c3-826a-c9b1f0316e32 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:37.768289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400baa4-18b0-42c6-acdb-509ec9bc26f5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:37.806770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49359a1e-bf26-4500-8280-80efcc09295e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:37.807847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49359a1e-bf26-4500-8280-80efcc09295e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:37.808370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49359a1e-bf26-4500-8280-80efcc09295e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:37.808855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49359a1e-bf26-4500-8280-80efcc09295e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:37.817900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49359a1e-bf26-4500-8280-80efcc09295e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:37.819662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49359a1e-bf26-4500-8280-80efcc09295e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:37.820288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e1adb-7535-47c3-826a-c9b1f0316e32 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-49359a1e-bf26-4500-8280-80efcc09295e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:37.832884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:37.833852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 05:14:37.834944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:37.837370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 304/596] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:37.842922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49359a1e-bf26-4500-8280-80efcc09295e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:37.850593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:37.862021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072ab19-b082-420a-a64b-9377b444eb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:37.863092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072ab19-b082-420a-a64b-9377b444eb02 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:37.863465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072ab19-b082-420a-a64b-9377b444eb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:37.863831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072ab19-b082-420a-a64b-9377b444eb02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:37.874927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072ab19-b082-420a-a64b-9377b444eb02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:37.876480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072ab19-b082-420a-a64b-9377b444eb02 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-b072ab19-b082-420a-a64b-9377b444eb02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:37.892637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:37.893738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:37.894873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:37.894873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:37.894873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:37.895719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 293/597] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:37.904645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072ab19-b082-420a-a64b-9377b444eb02 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:38.008860 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49359a1e-bf26-4500-8280-80efcc09295e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:38.034865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:38.071128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072ab19-b082-420a-a64b-9377b444eb02 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:38.080675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc8de2-48b7-4f11-9974-53deefbed839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:38.081297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc8de2-48b7-4f11-9974-53deefbed839 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:38.081575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc8de2-48b7-4f11-9974-53deefbed839 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:38.081849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc8de2-48b7-4f11-9974-53deefbed839 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:38.082666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc8de2-48b7-4f11-9974-53deefbed839 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:38.082970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc8de2-48b7-4f11-9974-53deefbed839 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:38.084390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc8de2-48b7-4f11-9974-53deefbed839 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:38.117318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 305/598] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Sep 21 05:14:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:38.123463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc8de2-48b7-4f11-9974-53deefbed839 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:38.143079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52be9e-a27b-417a-b5d0-f564f8a41419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:38.143691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52be9e-a27b-417a-b5d0-f564f8a41419 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:38.143992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52be9e-a27b-417a-b5d0-f564f8a41419 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:38.144308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52be9e-a27b-417a-b5d0-f564f8a41419 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:38.144851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52be9e-a27b-417a-b5d0-f564f8a41419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:38.145151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52be9e-a27b-417a-b5d0-f564f8a41419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:38.146551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52be9e-a27b-417a-b5d0-f564f8a41419 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:38.164517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 294/599] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:38.171156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52be9e-a27b-417a-b5d0-f564f8a41419 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:38.214948 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:38.314908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52be9e-a27b-417a-b5d0-f564f8a41419 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:38.324138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc8de2-48b7-4f11-9974-53deefbed839 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:38.361047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e583c5-1ddb-42c3-8224-b54dfe735be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:38.362066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e583c5-1ddb-42c3-8224-b54dfe735be6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:38.362844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e583c5-1ddb-42c3-8224-b54dfe735be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:38.363371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e583c5-1ddb-42c3-8224-b54dfe735be6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:38.374051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e583c5-1ddb-42c3-8224-b54dfe735be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:38.375712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e583c5-1ddb-42c3-8224-b54dfe735be6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:38.384053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdb860-13e9-4de1-a55d-015a2daf9df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:38.384684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdb860-13e9-4de1-a55d-015a2daf9df7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:38.384964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdb860-13e9-4de1-a55d-015a2daf9df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:38.385238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdb860-13e9-4de1-a55d-015a2daf9df7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c0e583c5-1ddb-42c3-8224-b54dfe735be6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:38.392926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:38.393831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:38.394741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:38.394741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:38.394741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:38.396841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 306/600] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:38.401335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdb860-13e9-4de1-a55d-015a2daf9df7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:38.402399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e583c5-1ddb-42c3-8224-b54dfe735be6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:38.405371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdb860-13e9-4de1-a55d-015a2daf9df7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-b9cdb860-13e9-4de1-a55d-015a2daf9df7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:38.430291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:38.431221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:38.432089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:38.432089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:38.432089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:38.432089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:38.432089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:38.432089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:38.432089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:38.432089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:38.438886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 295/601] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:38.444954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdb860-13e9-4de1-a55d-015a2daf9df7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:38.454285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:38.539277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e583c5-1ddb-42c3-8224-b54dfe735be6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:38.576001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbae78-61e3-4a10-a588-fb8692db6b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:38.576537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbae78-61e3-4a10-a588-fb8692db6b18 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:38.576819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbae78-61e3-4a10-a588-fb8692db6b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:38.577097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbae78-61e3-4a10-a588-fb8692db6b18 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:38.577913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbae78-61e3-4a10-a588-fb8692db6b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:38.578245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbae78-61e3-4a10-a588-fb8692db6b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:38.579620 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbae78-61e3-4a10-a588-fb8692db6b18 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:38.593330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 307/602] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:38.598987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbae78-61e3-4a10-a588-fb8692db6b18 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:38.638422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:38.644405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdb860-13e9-4de1-a55d-015a2daf9df7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:38.708372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb408cc-2855-4367-ae4d-39e67be0fb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:38.709362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb408cc-2855-4367-ae4d-39e67be0fb28 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:38.709857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb408cc-2855-4367-ae4d-39e67be0fb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:38.710349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb408cc-2855-4367-ae4d-39e67be0fb28 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:38.711290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb408cc-2855-4367-ae4d-39e67be0fb28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:38.711790 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb408cc-2855-4367-ae4d-39e67be0fb28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:38.712335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:38.713432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb408cc-2855-4367-ae4d-39e67be0fb28 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:38.728507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbae78-61e3-4a10-a588-fb8692db6b18 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:38.740762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 296/603] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:38.746360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb408cc-2855-4367-ae4d-39e67be0fb28 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:38.767236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86b02f-9a1a-491e-bf07-b040db89745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:38.767915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86b02f-9a1a-491e-bf07-b040db89745e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:38.768316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86b02f-9a1a-491e-bf07-b040db89745e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:38.768597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86b02f-9a1a-491e-bf07-b040db89745e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:38.776339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86b02f-9a1a-491e-bf07-b040db89745e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:38.777947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86b02f-9a1a-491e-bf07-b040db89745e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3c86b02f-9a1a-491e-bf07-b040db89745e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:38.790249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:38.791127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:38.792003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:38.792003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:38.792003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:38.792003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:38.792003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:38.792003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:38.792003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:38.792003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:38.792003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 308/604] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:38.797161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86b02f-9a1a-491e-bf07-b040db89745e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:38.893691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb408cc-2855-4367-ae4d-39e67be0fb28 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:38.947137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d86b7-6f97-4e72-b824-02d6e8325830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:38.947801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d86b7-6f97-4e72-b824-02d6e8325830 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:38.948083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d86b7-6f97-4e72-b824-02d6e8325830 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:38.948356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d86b7-6f97-4e72-b824-02d6e8325830 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/c07403facfaf4c71ab5324af6c44f52a/roles/99269efe3b2a430e9cc7912268d2be84` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:38.993863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86b02f-9a1a-491e-bf07-b040db89745e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:39.000180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d86b7-6f97-4e72-b824-02d6e8325830 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=c07403facfaf4c71ab5324af6c44f52a, role_id=99269efe3b2a430e9cc7912268d2be84)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:39.001901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d86b7-6f97-4e72-b824-02d6e8325830 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:39.034790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6993-2b87-4684-9cde-d0d58eddbd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:39.035410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6993-2b87-4684-9cde-d0d58eddbd47 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:39.035698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6993-2b87-4684-9cde-d0d58eddbd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:39.035985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6993-2b87-4684-9cde-d0d58eddbd47 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:39.036722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b6993-2b87-4684-9cde-d0d58eddbd47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:39.037036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b6993-2b87-4684-9cde-d0d58eddbd47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:39.038574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b6993-2b87-4684-9cde-d0d58eddbd47 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:39.045241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 297/605] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:14:38 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/c07403facfaf4c71ab5324af6c44f52a/roles/99269efe3b2a430e9cc7912268d2be84 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:14:39.051462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 309/606] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:39.057300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d86b7-6f97-4e72-b824-02d6e8325830 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:39.082359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:39.214822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a81b-9756-4882-935a-c8893464ffd7 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:39.215494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a81b-9756-4882-935a-c8893464ffd7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:39.215988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a81b-9756-4882-935a-c8893464ffd7 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:39.217906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 310/607] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:39.221230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d86b7-6f97-4e72-b824-02d6e8325830 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:39.224313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcfebd-c195-4b00-974d-fdb94358ca9d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:39.224890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcfebd-c195-4b00-974d-fdb94358ca9d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:39.225368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcfebd-c195-4b00-974d-fdb94358ca9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:39.255077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:39.261668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc9c12-ec3c-44f5-b703-de22a21f37db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:39.265956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc9c12-ec3c-44f5-b703-de22a21f37db None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:39.265956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc9c12-ec3c-44f5-b703-de22a21f37db None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:39.265956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc9c12-ec3c-44f5-b703-de22a21f37db None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/63a04998aafd40cc941283700898c92f/roles/99269efe3b2a430e9cc7912268d2be84` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:39.277962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-edfcfebd-c195-4b00-974d-fdb94358ca9d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:39.277962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-edfcfebd-c195-4b00-974d-fdb94358ca9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:39.278325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc9c12-ec3c-44f5-b703-de22a21f37db None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=63a04998aafd40cc941283700898c92f, role_id=99269efe3b2a430e9cc7912268d2be84)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:39.279243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc9c12-ec3c-44f5-b703-de22a21f37db None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:39.302574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 298/608] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:14:39 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/63a04998aafd40cc941283700898c92f/roles/99269efe3b2a430e9cc7912268d2be84 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:14:39.367672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 311/609] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:14:39.658057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc9c12-ec3c-44f5-b703-de22a21f37db None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:39.734497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:39.795178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc9c12-ec3c-44f5-b703-de22a21f37db None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:39.841960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488369e6-c620-4693-b682-3b15a59c5095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:39.842524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488369e6-c620-4693-b682-3b15a59c5095 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:39.842800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488369e6-c620-4693-b682-3b15a59c5095 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:39.843067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488369e6-c620-4693-b682-3b15a59c5095 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:39.858048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488369e6-c620-4693-b682-3b15a59c5095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:39.860317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556688b6-26f4-4fe8-a6bf-d0f51cac643f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:39.860770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556688b6-26f4-4fe8-a6bf-d0f51cac643f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:39.860855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556688b6-26f4-4fe8-a6bf-d0f51cac643f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:39.862467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 312/610] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-488369e6-c620-4693-b682-3b15a59c5095 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:39.863222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:39.864149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:39.864989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:39.865641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 299/611] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:39.870298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515672-3aa0-48a8-9345-e6d342850829 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:39.870580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515672-3aa0-48a8-9345-e6d342850829 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:39.870859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515672-3aa0-48a8-9345-e6d342850829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:39.871024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488369e6-c620-4693-b682-3b15a59c5095 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:39.911592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-32515672-3aa0-48a8-9345-e6d342850829 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:39.913281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-32515672-3aa0-48a8-9345-e6d342850829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:39.974277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488369e6-c620-4693-b682-3b15a59c5095 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:39.992876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:40.010147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf61f7b-ddc6-4d72-8458-fcaa5dc6be84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:40.010754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf61f7b-ddc6-4d72-8458-fcaa5dc6be84 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:40.011055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf61f7b-ddc6-4d72-8458-fcaa5dc6be84 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:40.011336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf61f7b-ddc6-4d72-8458-fcaa5dc6be84 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:40.016123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf61f7b-ddc6-4d72-8458-fcaa5dc6be84 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:40.016123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf61f7b-ddc6-4d72-8458-fcaa5dc6be84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:40.016123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf61f7b-ddc6-4d72-8458-fcaa5dc6be84 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:40.024948 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 300/612] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:40.034110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf61f7b-ddc6-4d72-8458-fcaa5dc6be84 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:40.053264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 313/613] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:40.156726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf61f7b-ddc6-4d72-8458-fcaa5dc6be84 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:40.190298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a7e0a-d2a8-4170-90f6-27de8bbf0068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:40.190930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a7e0a-d2a8-4170-90f6-27de8bbf0068 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:40.191206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a7e0a-d2a8-4170-90f6-27de8bbf0068 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:40.191481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a7e0a-d2a8-4170-90f6-27de8bbf0068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:40.211426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a7e0a-d2a8-4170-90f6-27de8bbf0068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:40.212938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a7e0a-d2a8-4170-90f6-27de8bbf0068 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-a32a7e0a-d2a8-4170-90f6-27de8bbf0068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:40.230008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:40.231048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:40.231920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:40.231920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:40.231920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:40.231920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 301/614] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:40.238399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515672-3aa0-48a8-9345-e6d342850829 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:40.315431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:40.358808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a7e0a-d2a8-4170-90f6-27de8bbf0068 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:40.367070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515672-3aa0-48a8-9345-e6d342850829 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:40.413522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:40.444807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48475b1-ae83-428b-9767-b407e84ed9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:40.445415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48475b1-ae83-428b-9767-b407e84ed9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:40.445689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48475b1-ae83-428b-9767-b407e84ed9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:40.445970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48475b1-ae83-428b-9767-b407e84ed9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:40.446608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48475b1-ae83-428b-9767-b407e84ed9e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:40.446908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48475b1-ae83-428b-9767-b407e84ed9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:40.448248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48475b1-ae83-428b-9767-b407e84ed9e5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:40.470869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 314/615] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:40.475733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48475b1-ae83-428b-9767-b407e84ed9e5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:40.535029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a7e0a-d2a8-4170-90f6-27de8bbf0068 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:40.579445 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd564500-b697-49a1-aa3d-6c5dc2e67152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:40.580068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd564500-b697-49a1-aa3d-6c5dc2e67152 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:40.580343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd564500-b697-49a1-aa3d-6c5dc2e67152 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:40.580612 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd564500-b697-49a1-aa3d-6c5dc2e67152 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:40.601140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd564500-b697-49a1-aa3d-6c5dc2e67152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-cd564500-b697-49a1-aa3d-6c5dc2e67152 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:40.606453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:40.607379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:40.608569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 302/616] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:40.609246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df037b4-2c85-4b78-8518-79e49738fee3 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:40.609246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df037b4-2c85-4b78-8518-79e49738fee3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:40.609652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df037b4-2c85-4b78-8518-79e49738fee3 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:40.615991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 303/617] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:40.616526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48475b1-ae83-428b-9767-b407e84ed9e5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:40.616743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df037b4-2c85-4b78-8518-79e49738fee3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:40.642383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:40.655272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb6610-7fdc-4542-8cc8-c84474daec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:40.656182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb6610-7fdc-4542-8cc8-c84474daec8f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:40.656687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb6610-7fdc-4542-8cc8-c84474daec8f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:40.657158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb6610-7fdc-4542-8cc8-c84474daec8f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:40.665055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb6610-7fdc-4542-8cc8-c84474daec8f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:40.667254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb6610-7fdc-4542-8cc8-c84474daec8f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-accb6610-7fdc-4542-8cc8-c84474daec8f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:40.679628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:40.680581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:14:40.681516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:40.688080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 315/618] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:40.688080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8b7ee-f4d2-474d-97c7-e9ea40eaafa8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:40.688080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8b7ee-f4d2-474d-97c7-e9ea40eaafa8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:40.688080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8b7ee-f4d2-474d-97c7-e9ea40eaafa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:40.707783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-cbf8b7ee-f4d2-474d-97c7-e9ea40eaafa8 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:40.709294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cbf8b7ee-f4d2-474d-97c7-e9ea40eaafa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:40.719289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:40.730347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df037b4-2c85-4b78-8518-79e49738fee3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:40.766081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511b1c8-9109-4cc7-96ba-8512079748fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:40.766685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511b1c8-9109-4cc7-96ba-8512079748fb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:40.766961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511b1c8-9109-4cc7-96ba-8512079748fb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:40.767380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511b1c8-9109-4cc7-96ba-8512079748fb None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:40.768046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511b1c8-9109-4cc7-96ba-8512079748fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:40.768342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511b1c8-9109-4cc7-96ba-8512079748fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:40.770031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511b1c8-9109-4cc7-96ba-8512079748fb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:40.785912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 304/619] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:40.786796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511b1c8-9109-4cc7-96ba-8512079748fb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:40.788407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 316/620] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:40.789523 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8b7ee-f4d2-474d-97c7-e9ea40eaafa8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:40.916583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511b1c8-9109-4cc7-96ba-8512079748fb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:40.935972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8b7ee-f4d2-474d-97c7-e9ea40eaafa8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:40.964865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:40.987975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8317b40-ae41-41b0-9505-cc8e9ae2d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:40.988608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8317b40-ae41-41b0-9505-cc8e9ae2d99e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:40.988851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8317b40-ae41-41b0-9505-cc8e9ae2d99e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:40.989120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8317b40-ae41-41b0-9505-cc8e9ae2d99e None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:40.989840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8317b40-ae41-41b0-9505-cc8e9ae2d99e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:40.990168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8317b40-ae41-41b0-9505-cc8e9ae2d99e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:40.991759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8317b40-ae41-41b0-9505-cc8e9ae2d99e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.005515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59151a-16fc-4014-9576-b5935eafaec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.006228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59151a-16fc-4014-9576-b5935eafaec8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.006506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59151a-16fc-4014-9576-b5935eafaec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.006777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59151a-16fc-4014-9576-b5935eafaec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.025066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 305/621] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Sep 21 05:14:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:41.025949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59151a-16fc-4014-9576-b5935eafaec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:41.027501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59151a-16fc-4014-9576-b5935eafaec8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.031405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8317b40-ae41-41b0-9505-cc8e9ae2d99e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.051775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-6f59151a-16fc-4014-9576-b5935eafaec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:41.059519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:41.060455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:41.061710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:41.061710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:41.061710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:41.061710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 317/622] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:41.070831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59151a-16fc-4014-9576-b5935eafaec8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.162517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8317b40-ae41-41b0-9505-cc8e9ae2d99e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:41.199499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a137ac-daf7-4a7f-88e5-383560786fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.199499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a137ac-daf7-4a7f-88e5-383560786fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.199499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a137ac-daf7-4a7f-88e5-383560786fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.199499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a137ac-daf7-4a7f-88e5-383560786fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.210083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59151a-16fc-4014-9576-b5935eafaec8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:41.211618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a137ac-daf7-4a7f-88e5-383560786fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:41.213167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a137ac-daf7-4a7f-88e5-383560786fc5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c4a137ac-daf7-4a7f-88e5-383560786fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:41.228654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:41.229590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:41.230800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:41.230800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:41.230800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:41.230800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 306/623] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:41.231997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a137ac-daf7-4a7f-88e5-383560786fc5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.251636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762c6f7-5ad2-4126-bbb4-956bccbc5139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.252243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762c6f7-5ad2-4126-bbb4-956bccbc5139 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.252520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762c6f7-5ad2-4126-bbb4-956bccbc5139 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.253037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762c6f7-5ad2-4126-bbb4-956bccbc5139 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.253673 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762c6f7-5ad2-4126-bbb4-956bccbc5139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:41.254175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762c6f7-5ad2-4126-bbb4-956bccbc5139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:41.256618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762c6f7-5ad2-4126-bbb4-956bccbc5139 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.279508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 318/624] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:41.279508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762c6f7-5ad2-4126-bbb4-956bccbc5139 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.302572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:41.319865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:41.347120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a137ac-daf7-4a7f-88e5-383560786fc5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:41.386422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f3ccd-ac45-4524-adcb-8c59c8f54135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.387100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f3ccd-ac45-4524-adcb-8c59c8f54135 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.387379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f3ccd-ac45-4524-adcb-8c59c8f54135 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.387649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f3ccd-ac45-4524-adcb-8c59c8f54135 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.403223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f3ccd-ac45-4524-adcb-8c59c8f54135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-ee6f3ccd-ac45-4524-adcb-8c59c8f54135 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:41.413524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:41.414559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:41.415376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:41.421513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 307/625] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:41.422501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f3ccd-ac45-4524-adcb-8c59c8f54135 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.422724 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762c6f7-5ad2-4126-bbb4-956bccbc5139 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:41.472104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8858cb-d7c4-4f69-b5fa-6c3d8873b224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.473438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8858cb-d7c4-4f69-b5fa-6c3d8873b224 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.474171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8858cb-d7c4-4f69-b5fa-6c3d8873b224 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.474598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8858cb-d7c4-4f69-b5fa-6c3d8873b224 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.475433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8858cb-d7c4-4f69-b5fa-6c3d8873b224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:41.476146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8858cb-d7c4-4f69-b5fa-6c3d8873b224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:41.478621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8858cb-d7c4-4f69-b5fa-6c3d8873b224 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.497125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 319/626] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:41.498312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8858cb-d7c4-4f69-b5fa-6c3d8873b224 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.537408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f3ccd-ac45-4524-adcb-8c59c8f54135 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:41.565672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:41.582799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093908a-3b95-4e1d-9f1e-b3615b69e03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.589077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093908a-3b95-4e1d-9f1e-b3615b69e03d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.589077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093908a-3b95-4e1d-9f1e-b3615b69e03d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.589077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093908a-3b95-4e1d-9f1e-b3615b69e03d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.594864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093908a-3b95-4e1d-9f1e-b3615b69e03d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:41.596732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093908a-3b95-4e1d-9f1e-b3615b69e03d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3093908a-3b95-4e1d-9f1e-b3615b69e03d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:41.608163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:41.609140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 308/627] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:41.610250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093908a-3b95-4e1d-9f1e-b3615b69e03d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.617407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:41.635176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8858cb-d7c4-4f69-b5fa-6c3d8873b224 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:41.672589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a82a2-a2fb-40ff-80b8-b7eea769b070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.673195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a82a2-a2fb-40ff-80b8-b7eea769b070 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.673474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a82a2-a2fb-40ff-80b8-b7eea769b070 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.673744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a82a2-a2fb-40ff-80b8-b7eea769b070 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.674420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a82a2-a2fb-40ff-80b8-b7eea769b070 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:41.674721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a82a2-a2fb-40ff-80b8-b7eea769b070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:41.676326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a82a2-a2fb-40ff-80b8-b7eea769b070 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.688901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 320/628] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:41.688901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e84183-3193-4fce-a55b-679f87a5323c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.688901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e84183-3193-4fce-a55b-679f87a5323c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.688901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e84183-3193-4fce-a55b-679f87a5323c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.688901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 321/629] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:41.688901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e84183-3193-4fce-a55b-679f87a5323c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.726463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093908a-3b95-4e1d-9f1e-b3615b69e03d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:41.763469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440b3c4-2db0-4e17-8c32-0179c6c3e9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.764102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440b3c4-2db0-4e17-8c32-0179c6c3e9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.764385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440b3c4-2db0-4e17-8c32-0179c6c3e9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.764660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440b3c4-2db0-4e17-8c32-0179c6c3e9d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.771420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440b3c4-2db0-4e17-8c32-0179c6c3e9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:41.772874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440b3c4-2db0-4e17-8c32-0179c6c3e9d3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-a440b3c4-2db0-4e17-8c32-0179c6c3e9d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:41.784567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:41.785488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 309/630] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:41.786414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440b3c4-2db0-4e17-8c32-0179c6c3e9d3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.803621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e84183-3193-4fce-a55b-679f87a5323c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:41.838824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:41.843950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea30b1-9c14-4106-805e-83c1332105d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.844726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea30b1-9c14-4106-805e-83c1332105d4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.845190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea30b1-9c14-4106-805e-83c1332105d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.845467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea30b1-9c14-4106-805e-83c1332105d4 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.846223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea30b1-9c14-4106-805e-83c1332105d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:41.846530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea30b1-9c14-4106-805e-83c1332105d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:41.847940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea30b1-9c14-4106-805e-83c1332105d4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.867144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 322/631] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:41.868367 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227c753-7a18-4f98-a655-04b302d51b3b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.868657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227c753-7a18-4f98-a655-04b302d51b3b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.868935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227c753-7a18-4f98-a655-04b302d51b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.881792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:41.888671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2227c753-7a18-4f98-a655-04b302d51b3b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:41.889868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2227c753-7a18-4f98-a655-04b302d51b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:41.904921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440b3c4-2db0-4e17-8c32-0179c6c3e9d3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:41.938836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68501416-f900-4569-87db-09fa9d92a883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:41.939459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68501416-f900-4569-87db-09fa9d92a883 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:41.939735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68501416-f900-4569-87db-09fa9d92a883 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:41.940009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68501416-f900-4569-87db-09fa9d92a883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:41.950256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68501416-f900-4569-87db-09fa9d92a883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:41.951696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68501416-f900-4569-87db-09fa9d92a883 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:41.961986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 323/632] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:41.972310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227c753-7a18-4f98-a655-04b302d51b3b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-68501416-f900-4569-87db-09fa9d92a883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:41.972419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:41.973452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:41.974563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:41.974563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:41.974563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:41.974563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 310/633] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:41.974563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68501416-f900-4569-87db-09fa9d92a883 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.072181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:42.075263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227c753-7a18-4f98-a655-04b302d51b3b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:42.079762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68501416-f900-4569-87db-09fa9d92a883 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:42.113766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2836b0-0f9f-4918-aa90-da372ed59fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:42.113973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6bc9d-29f9-43e2-8b04-7a576dea579c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:42.114416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6bc9d-29f9-43e2-8b04-7a576dea579c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.114563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2836b0-0f9f-4918-aa90-da372ed59fad None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.114708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6bc9d-29f9-43e2-8b04-7a576dea579c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.114951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2836b0-0f9f-4918-aa90-da372ed59fad None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.115065 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6bc9d-29f9-43e2-8b04-7a576dea579c None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.115321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2836b0-0f9f-4918-aa90-da372ed59fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.115639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6bc9d-29f9-43e2-8b04-7a576dea579c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:42.115947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6bc9d-29f9-43e2-8b04-7a576dea579c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:42.117363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6bc9d-29f9-43e2-8b04-7a576dea579c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:42.120135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:42.131732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 324/634] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:42.133423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78832e-fefb-45f2-9560-a2646f0004e8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.133856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78832e-fefb-45f2-9560-a2646f0004e8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.134157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78832e-fefb-45f2-9560-a2646f0004e8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.135498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 325/635] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:42.136023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78832e-fefb-45f2-9560-a2646f0004e8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.190621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2836b0-0f9f-4918-aa90-da372ed59fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:42.192156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2836b0-0f9f-4918-aa90-da372ed59fad None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:42.249487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78832e-fefb-45f2-9560-a2646f0004e8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:42.282641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac11f3c-8ea7-4d81-8cfd-138d88d78b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:42.283244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac11f3c-8ea7-4d81-8cfd-138d88d78b60 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.283526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac11f3c-8ea7-4d81-8cfd-138d88d78b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.283809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac11f3c-8ea7-4d81-8cfd-138d88d78b60 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.284441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac11f3c-8ea7-4d81-8cfd-138d88d78b60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:42.284749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac11f3c-8ea7-4d81-8cfd-138d88d78b60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:42.286188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac11f3c-8ea7-4d81-8cfd-138d88d78b60 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:42.286628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:42.293504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 311/636] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:42.294304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2836b0-0f9f-4918-aa90-da372ed59fad None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.297631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 326/637] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:42.298861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a04f5-2514-460d-9248-e4cc86659c02 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.299146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a04f5-2514-460d-9248-e4cc86659c02 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.299417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a04f5-2514-460d-9248-e4cc86659c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.317236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-808a04f5-2514-460d-9248-e4cc86659c02 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:42.318497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-808a04f5-2514-460d-9248-e4cc86659c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:42.369253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:42.397964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 327/638] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:42.397964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a04f5-2514-460d-9248-e4cc86659c02 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.398254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2836b0-0f9f-4918-aa90-da372ed59fad None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:42.428709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d6f8-20d1-448a-934a-6bf516a1bc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:42.429697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d6f8-20d1-448a-934a-6bf516a1bc82 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.430023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d6f8-20d1-448a-934a-6bf516a1bc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.430303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d6f8-20d1-448a-934a-6bf516a1bc82 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/ee0fe35d9e304ee495998c93c6909fa8/roles/99269efe3b2a430e9cc7912268d2be84` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.445826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4d6f8-20d1-448a-934a-6bf516a1bc82 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=ee0fe35d9e304ee495998c93c6909fa8, role_id=99269efe3b2a430e9cc7912268d2be84)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:42.447308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4d6f8-20d1-448a-934a-6bf516a1bc82 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:42.461819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 312/639] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:14:42 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/ee0fe35d9e304ee495998c93c6909fa8/roles/99269efe3b2a430e9cc7912268d2be84 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:14:42.462812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d6f8-20d1-448a-934a-6bf516a1bc82 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.497903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a04f5-2514-460d-9248-e4cc86659c02 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:42.546482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:42.549971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6b47b-21f3-49d0-a51d-ec16a611ff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:42.550928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6b47b-21f3-49d0-a51d-ec16a611ff16 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.551541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6b47b-21f3-49d0-a51d-ec16a611ff16 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.552019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6b47b-21f3-49d0-a51d-ec16a611ff16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.570797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6b47b-21f3-49d0-a51d-ec16a611ff16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-fab6b47b-21f3-49d0-a51d-ec16a611ff16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:42.576254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:42.577350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:42.578219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:42.582327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 328/640] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:42.583276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6b47b-21f3-49d0-a51d-ec16a611ff16 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.640055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d6f8-20d1-448a-934a-6bf516a1bc82 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:42.649582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:42.678930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca65d6-e910-43e7-afa2-0e9502da76af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:42.679544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca65d6-e910-43e7-afa2-0e9502da76af None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.679820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca65d6-e910-43e7-afa2-0e9502da76af None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.680091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca65d6-e910-43e7-afa2-0e9502da76af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.691115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca65d6-e910-43e7-afa2-0e9502da76af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:42.692572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca65d6-e910-43e7-afa2-0e9502da76af None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-63ca65d6-e910-43e7-afa2-0e9502da76af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:42.708304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:42.709243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:42.710795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:42.710795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:42.710795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:42.710795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 313/641] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:42.710795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca65d6-e910-43e7-afa2-0e9502da76af None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.720529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6b47b-21f3-49d0-a51d-ec16a611ff16 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:42.756310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503cf2c-a956-48e1-b8a7-e6edbb74dfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:42.756942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503cf2c-a956-48e1-b8a7-e6edbb74dfd7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.757338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503cf2c-a956-48e1-b8a7-e6edbb74dfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.757618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503cf2c-a956-48e1-b8a7-e6edbb74dfd7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.766700 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503cf2c-a956-48e1-b8a7-e6edbb74dfd7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:42.768651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503cf2c-a956-48e1-b8a7-e6edbb74dfd7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5503cf2c-a956-48e1-b8a7-e6edbb74dfd7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:42.780768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:42.781790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 329/642] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Sep 21 05:14:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:42.787359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503cf2c-a956-48e1-b8a7-e6edbb74dfd7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.821518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:42.827474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca65d6-e910-43e7-afa2-0e9502da76af None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:42.865749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff212d-e4e7-4c69-b373-fb8698f5fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:42.866497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff212d-e4e7-4c69-b373-fb8698f5fdbf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.866770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff212d-e4e7-4c69-b373-fb8698f5fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.867037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff212d-e4e7-4c69-b373-fb8698f5fdbf None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.867694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff212d-e4e7-4c69-b373-fb8698f5fdbf None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:42.868113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff212d-e4e7-4c69-b373-fb8698f5fdbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:42.869467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff212d-e4e7-4c69-b373-fb8698f5fdbf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:42.883070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 314/643] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:42.883070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff212d-e4e7-4c69-b373-fb8698f5fdbf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.896056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503cf2c-a956-48e1-b8a7-e6edbb74dfd7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:42.902992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:42.931506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3e6cb-dd9e-4590-981e-6662fba0d9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:42.932133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3e6cb-dd9e-4590-981e-6662fba0d9cb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:42.932409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3e6cb-dd9e-4590-981e-6662fba0d9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:42.932674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3e6cb-dd9e-4590-981e-6662fba0d9cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:42.943274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3e6cb-dd9e-4590-981e-6662fba0d9cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-29c3e6cb-dd9e-4590-981e-6662fba0d9cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:42.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:42.949247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 330/644] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:42.955302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3e6cb-dd9e-4590-981e-6662fba0d9cb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:42.994926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff212d-e4e7-4c69-b373-fb8698f5fdbf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.029687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce4e64-6205-4e05-91a2-6b6767c17cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.030308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce4e64-6205-4e05-91a2-6b6767c17cba None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.030589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce4e64-6205-4e05-91a2-6b6767c17cba None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.030859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce4e64-6205-4e05-91a2-6b6767c17cba None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.031487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce4e64-6205-4e05-91a2-6b6767c17cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:43.031784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce4e64-6205-4e05-91a2-6b6767c17cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.033132 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce4e64-6205-4e05-91a2-6b6767c17cba None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.049811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 315/645] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:43.049811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce4e64-6205-4e05-91a2-6b6767c17cba None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.067094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3e6cb-dd9e-4590-981e-6662fba0d9cb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.071470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:43.103218 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faeda47-3ba9-4cdb-b325-34a43e3aa096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.103838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faeda47-3ba9-4cdb-b325-34a43e3aa096 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.104119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faeda47-3ba9-4cdb-b325-34a43e3aa096 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.104396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faeda47-3ba9-4cdb-b325-34a43e3aa096 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.105122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faeda47-3ba9-4cdb-b325-34a43e3aa096 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:43.105430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faeda47-3ba9-4cdb-b325-34a43e3aa096 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.107468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faeda47-3ba9-4cdb-b325-34a43e3aa096 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.130513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 331/646] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Sep 21 05:14:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:43.130513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faeda47-3ba9-4cdb-b325-34a43e3aa096 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.153275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:43.164645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce4e64-6205-4e05-91a2-6b6767c17cba None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.200711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3dcc4-9030-4302-918b-3edd3d0eee90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.201359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3dcc4-9030-4302-918b-3edd3d0eee90 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.201684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3dcc4-9030-4302-918b-3edd3d0eee90 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.201989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3dcc4-9030-4302-918b-3edd3d0eee90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.212512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3dcc4-9030-4302-918b-3edd3d0eee90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.213964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3dcc4-9030-4302-918b-3edd3d0eee90 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3cd3dcc4-9030-4302-918b-3edd3d0eee90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:43.228782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:43.229954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:43.231602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:43.231602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:43.231602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:43.231602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 316/647] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:43.231602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3dcc4-9030-4302-918b-3edd3d0eee90 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.246219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faeda47-3ba9-4cdb-b325-34a43e3aa096 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.282838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83144516-dde5-4b18-a3f7-b244fdb4fc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.283666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83144516-dde5-4b18-a3f7-b244fdb4fc23 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.284093 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83144516-dde5-4b18-a3f7-b244fdb4fc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.284519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83144516-dde5-4b18-a3f7-b244fdb4fc23 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.285339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83144516-dde5-4b18-a3f7-b244fdb4fc23 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:43.285641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83144516-dde5-4b18-a3f7-b244fdb4fc23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.287022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83144516-dde5-4b18-a3f7-b244fdb4fc23 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.296198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 332/648] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:43.296885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83144516-dde5-4b18-a3f7-b244fdb4fc23 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.311297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:43.345020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3dcc4-9030-4302-918b-3edd3d0eee90 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.383817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d3673-f208-4681-8656-9adf5c19da8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.384436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d3673-f208-4681-8656-9adf5c19da8e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.384712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d3673-f208-4681-8656-9adf5c19da8e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.384981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d3673-f208-4681-8656-9adf5c19da8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.392516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d3673-f208-4681-8656-9adf5c19da8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.393986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d3673-f208-4681-8656-9adf5c19da8e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.395784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-649d3673-f208-4681-8656-9adf5c19da8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:43.407222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:43.408113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 317/649] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:43.409003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d3673-f208-4681-8656-9adf5c19da8e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.417873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83144516-dde5-4b18-a3f7-b244fdb4fc23 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.454266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f39a29-04e7-44d1-9e9c-df02b861205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.454869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f39a29-04e7-44d1-9e9c-df02b861205c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.455151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f39a29-04e7-44d1-9e9c-df02b861205c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.455427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f39a29-04e7-44d1-9e9c-df02b861205c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.516995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d3673-f208-4681-8656-9adf5c19da8e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.525734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f39a29-04e7-44d1-9e9c-df02b861205c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.527274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f39a29-04e7-44d1-9e9c-df02b861205c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.548487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:43.554357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3799642-d9fd-47d7-9d0a-1ff38092d7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.555358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3799642-d9fd-47d7-9d0a-1ff38092d7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.555630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3799642-d9fd-47d7-9d0a-1ff38092d7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.555897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3799642-d9fd-47d7-9d0a-1ff38092d7c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.556522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3799642-d9fd-47d7-9d0a-1ff38092d7c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:43.556820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3799642-d9fd-47d7-9d0a-1ff38092d7c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.558357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3799642-d9fd-47d7-9d0a-1ff38092d7c4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.575687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 318/650] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:43.575687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3799642-d9fd-47d7-9d0a-1ff38092d7c4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.620476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:43.644789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 333/651] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:43.644789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f39a29-04e7-44d1-9e9c-df02b861205c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.686389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3799642-d9fd-47d7-9d0a-1ff38092d7c4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.722505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f85e14-2baa-4cff-9da5-76d48d292d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.723137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f85e14-2baa-4cff-9da5-76d48d292d46 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.723418 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f85e14-2baa-4cff-9da5-76d48d292d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.723690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f85e14-2baa-4cff-9da5-76d48d292d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.734453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f85e14-2baa-4cff-9da5-76d48d292d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.736062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f85e14-2baa-4cff-9da5-76d48d292d46 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-60f85e14-2baa-4cff-9da5-76d48d292d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:43.751559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:43.752528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:43.753423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:43.753423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:43.753423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:43.753601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f39a29-04e7-44d1-9e9c-df02b861205c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.757406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 319/652] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:43.758274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f85e14-2baa-4cff-9da5-76d48d292d46 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.794145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701912b-e69d-4353-985b-486906a15ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.794755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701912b-e69d-4353-985b-486906a15ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.795037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701912b-e69d-4353-985b-486906a15ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.795341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701912b-e69d-4353-985b-486906a15ca5 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.796030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701912b-e69d-4353-985b-486906a15ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:43.796340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701912b-e69d-4353-985b-486906a15ca5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.797780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701912b-e69d-4353-985b-486906a15ca5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.804038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:43.814473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 334/653] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:43.814473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701912b-e69d-4353-985b-486906a15ca5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.862317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:43.867596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f85e14-2baa-4cff-9da5-76d48d292d46 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.901610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cc7f5-aa7d-47ff-b038-255880bd817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.902267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cc7f5-aa7d-47ff-b038-255880bd817c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.903155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cc7f5-aa7d-47ff-b038-255880bd817c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.903724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cc7f5-aa7d-47ff-b038-255880bd817c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.911038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cc7f5-aa7d-47ff-b038-255880bd817c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.912509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cc7f5-aa7d-47ff-b038-255880bd817c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.922382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701912b-e69d-4353-985b-486906a15ca5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3b5cc7f5-aa7d-47ff-b038-255880bd817c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:43.923708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:43.924651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:43.925564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 320/654] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:43.926211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cc7f5-aa7d-47ff-b038-255880bd817c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:43.962280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddee412-3d40-41c9-a300-536f2f8c45dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:43.962834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddee412-3d40-41c9-a300-536f2f8c45dc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:43.963112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddee412-3d40-41c9-a300-536f2f8c45dc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:43.963387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddee412-3d40-41c9-a300-536f2f8c45dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:43.974954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddee412-3d40-41c9-a300-536f2f8c45dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:43.976387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddee412-3d40-41c9-a300-536f2f8c45dc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-6ddee412-3d40-41c9-a300-536f2f8c45dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:43.990912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:43.991872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:43.992739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:43.992739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:43.992739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:43.992739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 335/655] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:43.992972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddee412-3d40-41c9-a300-536f2f8c45dc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:44.039784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cc7f5-aa7d-47ff-b038-255880bd817c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:44.042094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:44.075604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2085e57-27a9-4df0-b236-fe35516cc8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:44.076210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2085e57-27a9-4df0-b236-fe35516cc8ee None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.076487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2085e57-27a9-4df0-b236-fe35516cc8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.076762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2085e57-27a9-4df0-b236-fe35516cc8ee None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.077402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2085e57-27a9-4df0-b236-fe35516cc8ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:44.077755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2085e57-27a9-4df0-b236-fe35516cc8ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:44.079155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2085e57-27a9-4df0-b236-fe35516cc8ee None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:44.094628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 321/656] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:44.100341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:44.103239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2085e57-27a9-4df0-b236-fe35516cc8ee None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:44.105104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddee412-3d40-41c9-a300-536f2f8c45dc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:44.143651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089915f-293e-434b-904b-cdc3d8141194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:44.144307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089915f-293e-434b-904b-cdc3d8141194 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.144589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089915f-293e-434b-904b-cdc3d8141194 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.144861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089915f-293e-434b-904b-cdc3d8141194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.145627 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089915f-293e-434b-904b-cdc3d8141194 None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=5305f8a5d2b540de91d3caedad454bfa) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:44.146376 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089915f-293e-434b-904b-cdc3d8141194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:44.147484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089915f-293e-434b-904b-cdc3d8141194 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:44.162234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 336/657] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:43 2022] GET /identity/v3/role_assignments?user.id=5305f8a5d2b540de91d3caedad454bfa&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:44.162234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089915f-293e-434b-904b-cdc3d8141194 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:44.227594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2085e57-27a9-4df0-b236-fe35516cc8ee None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:44.279899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600c005-cd01-42a9-9e14-d589fb4096b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:44.280629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600c005-cd01-42a9-9e14-d589fb4096b2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.280990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600c005-cd01-42a9-9e14-d589fb4096b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.281327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600c005-cd01-42a9-9e14-d589fb4096b2 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.281703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089915f-293e-434b-904b-cdc3d8141194 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:44.282229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600c005-cd01-42a9-9e14-d589fb4096b2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:44.282603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600c005-cd01-42a9-9e14-d589fb4096b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:44.285402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600c005-cd01-42a9-9e14-d589fb4096b2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:44.312474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 322/658] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Sep 21 05:14:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:44.313510 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600c005-cd01-42a9-9e14-d589fb4096b2 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:44.319684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d549ebd-2c11-4142-83d0-e21c3693f72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:44.320388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d549ebd-2c11-4142-83d0-e21c3693f72a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.320738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d549ebd-2c11-4142-83d0-e21c3693f72a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.321078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d549ebd-2c11-4142-83d0-e21c3693f72a None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.321804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d549ebd-2c11-4142-83d0-e21c3693f72a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:44.322221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d549ebd-2c11-4142-83d0-e21c3693f72a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:44.323686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d549ebd-2c11-4142-83d0-e21c3693f72a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:44.337985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 337/659] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:44.338874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d549ebd-2c11-4142-83d0-e21c3693f72a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:44.348664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:44.377251 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:44.439818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600c005-cd01-42a9-9e14-d589fb4096b2 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:44.484945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d549ebd-2c11-4142-83d0-e21c3693f72a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:44.485122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de240d5-6361-4723-b8d7-ad9cb9f4ef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:44.485706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de240d5-6361-4723-b8d7-ad9cb9f4ef64 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.486708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de240d5-6361-4723-b8d7-ad9cb9f4ef64 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.486708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de240d5-6361-4723-b8d7-ad9cb9f4ef64 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.498717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de240d5-6361-4723-b8d7-ad9cb9f4ef64 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:44.500253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de240d5-6361-4723-b8d7-ad9cb9f4ef64 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-6de240d5-6361-4723-b8d7-ad9cb9f4ef64 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:44.512694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:44.513689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 323/660] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:44.514661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de240d5-6361-4723-b8d7-ad9cb9f4ef64 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:44.531488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d96f8-2551-49df-8101-4068ae9978c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:44.532236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d96f8-2551-49df-8101-4068ae9978c0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.532634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d96f8-2551-49df-8101-4068ae9978c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.532976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d96f8-2551-49df-8101-4068ae9978c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.616250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d96f8-2551-49df-8101-4068ae9978c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:44.617923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d96f8-2551-49df-8101-4068ae9978c0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:44.620195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:44.635826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de240d5-6361-4723-b8d7-ad9cb9f4ef64 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:44.646822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:44.675341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78577-fef3-4a8c-a63e-ee9e96b532f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:44.675989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78577-fef3-4a8c-a63e-ee9e96b532f1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.676240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78577-fef3-4a8c-a63e-ee9e96b532f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.676517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78577-fef3-4a8c-a63e-ee9e96b532f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.684864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce78577-fef3-4a8c-a63e-ee9e96b532f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:44.686331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce78577-fef3-4a8c-a63e-ee9e96b532f1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-7ce78577-fef3-4a8c-a63e-ee9e96b532f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:44.697467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:44.698394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:44.699280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:44.699280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:44.699280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:44.699280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:44.699280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:44.699280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:44.699280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:44.699280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:44.699280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 324/661] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:44.699803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78577-fef3-4a8c-a63e-ee9e96b532f1 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:44.735753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 338/662] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:44.736954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6e857-ec98-4816-88d4-632d308be510 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.737234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6e857-ec98-4816-88d4-632d308be510 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.737504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6e857-ec98-4816-88d4-632d308be510 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.739155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 339/663] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:44.743727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6e857-ec98-4816-88d4-632d308be510 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:44.809805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78577-fef3-4a8c-a63e-ee9e96b532f1 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:44.843800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2d688-df80-486a-900f-ac20f9be290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:44.844433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2d688-df80-486a-900f-ac20f9be290e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.844693 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2d688-df80-486a-900f-ac20f9be290e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.844967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2d688-df80-486a-900f-ac20f9be290e None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.845689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2d688-df80-486a-900f-ac20f9be290e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:44.846032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2d688-df80-486a-900f-ac20f9be290e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:44.847023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6e857-ec98-4816-88d4-632d308be510 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:44.847461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2d688-df80-486a-900f-ac20f9be290e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:44.861929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:44.871086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 325/664] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Sep 21 05:14:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:44.871992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2d688-df80-486a-900f-ac20f9be290e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:44.872890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:44.887073 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc403501-c297-4852-a010-404ce1b23ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:44.887690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc403501-c297-4852-a010-404ce1b23ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.887974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc403501-c297-4852-a010-404ce1b23ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.888250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc403501-c297-4852-a010-404ce1b23ce5 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.888936 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc403501-c297-4852-a010-404ce1b23ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:44.889243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc403501-c297-4852-a010-404ce1b23ce5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:44.890711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc403501-c297-4852-a010-404ce1b23ce5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:44.902395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 340/665] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:44.908822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0df280-3bfb-4402-bfe3-e0486bd6321e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:44.909113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0df280-3bfb-4402-bfe3-e0486bd6321e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:44.909392 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0df280-3bfb-4402-bfe3-e0486bd6321e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:44.927564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ce0df280-3bfb-4402-bfe3-e0486bd6321e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:44.928762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ce0df280-3bfb-4402-bfe3-e0486bd6321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:44.974416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2d688-df80-486a-900f-ac20f9be290e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.000661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 341/666] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:45.001439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0df280-3bfb-4402-bfe3-e0486bd6321e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.018890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e8145-5dfa-4d28-a4f6-138b6c631698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:45.019911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e8145-5dfa-4d28-a4f6-138b6c631698 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:45.020413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e8145-5dfa-4d28-a4f6-138b6c631698 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:45.020881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e8145-5dfa-4d28-a4f6-138b6c631698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:45.033419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e8145-5dfa-4d28-a4f6-138b6c631698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:45.035791 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e8145-5dfa-4d28-a4f6-138b6c631698 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-095e8145-5dfa-4d28-a4f6-138b6c631698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:45.056461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:45.057636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:45.058768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:45.058768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:45.058768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:45.060632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 326/667] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:45.061548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e8145-5dfa-4d28-a4f6-138b6c631698 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.126916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0df280-3bfb-4402-bfe3-e0486bd6321e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.127910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:45.143220 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:45.167388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf63943-f610-46fa-a314-a6ddf485d1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:45.168762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf63943-f610-46fa-a314-a6ddf485d1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:45.169181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf63943-f610-46fa-a314-a6ddf485d1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:45.169589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf63943-f610-46fa-a314-a6ddf485d1a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:45.180647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e8145-5dfa-4d28-a4f6-138b6c631698 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.220691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17667f-a70c-4529-90ce-ae80cfd3bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:45.221770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17667f-a70c-4529-90ce-ae80cfd3bb9d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:45.222187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17667f-a70c-4529-90ce-ae80cfd3bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:45.222561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17667f-a70c-4529-90ce-ae80cfd3bb9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:45.223683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17667f-a70c-4529-90ce-ae80cfd3bb9d None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d9168d8bbf8b43299a7748c9faa099b1) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:45.224095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17667f-a70c-4529-90ce-ae80cfd3bb9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:45.226198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17667f-a70c-4529-90ce-ae80cfd3bb9d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:45.238075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 327/668] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/role_assignments?user.id=d9168d8bbf8b43299a7748c9faa099b1&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:45.239549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17667f-a70c-4529-90ce-ae80cfd3bb9d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.270796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf63943-f610-46fa-a314-a6ddf485d1a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:45.272503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf63943-f610-46fa-a314-a6ddf485d1a3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:45.378587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17667f-a70c-4529-90ce-ae80cfd3bb9d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.398246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:45.419017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0d126-1f1d-4e16-9a46-c144f27dfb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:45.419900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0d126-1f1d-4e16-9a46-c144f27dfb5a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:45.420175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0d126-1f1d-4e16-9a46-c144f27dfb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:45.420439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0d126-1f1d-4e16-9a46-c144f27dfb5a None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:45.421072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0d126-1f1d-4e16-9a46-c144f27dfb5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:45.421369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0d126-1f1d-4e16-9a46-c144f27dfb5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:45.422764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0d126-1f1d-4e16-9a46-c144f27dfb5a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:45.428386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 342/669] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:45.429408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf63943-f610-46fa-a314-a6ddf485d1a3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.438229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 328/670] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:45.439138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0d126-1f1d-4e16-9a46-c144f27dfb5a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.449266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:45.557788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0d126-1f1d-4e16-9a46-c144f27dfb5a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.564266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf63943-f610-46fa-a314-a6ddf485d1a3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.607336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d068-d249-4c08-87e6-fe24e88589d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:45.607973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d068-d249-4c08-87e6-fe24e88589d7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:45.608320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d068-d249-4c08-87e6-fe24e88589d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:45.608522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d068-d249-4c08-87e6-fe24e88589d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:45.616877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781f4cf-b91c-424e-937d-c23f716836db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:45.617789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781f4cf-b91c-424e-937d-c23f716836db None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:45.618712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781f4cf-b91c-424e-937d-c23f716836db None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:45.619318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781f4cf-b91c-424e-937d-c23f716836db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:45.620456 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076d068-d249-4c08-87e6-fe24e88589d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:45.621965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076d068-d249-4c08-87e6-fe24e88589d7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:45.631396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781f4cf-b91c-424e-937d-c23f716836db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1781f4cf-b91c-424e-937d-c23f716836db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:45.636515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:45.637462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:45.638312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0076d068-d249-4c08-87e6-fe24e88589d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:45.638986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:45.639840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:45.642664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:45.642664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:45.642664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:45.642664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 329/671] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:45.642664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d068-d249-4c08-87e6-fe24e88589d7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.645759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 343/672] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:45.646807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781f4cf-b91c-424e-937d-c23f716836db None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.678688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:45.744405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:45.757711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d068-d249-4c08-87e6-fe24e88589d7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.779853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781f4cf-b91c-424e-937d-c23f716836db None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.794020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba696967-fdf0-4475-96ac-38916e8f7a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:45.794687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba696967-fdf0-4475-96ac-38916e8f7a1f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:45.794929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba696967-fdf0-4475-96ac-38916e8f7a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:45.795207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba696967-fdf0-4475-96ac-38916e8f7a1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:45.802121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba696967-fdf0-4475-96ac-38916e8f7a1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:45.803582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba696967-fdf0-4475-96ac-38916e8f7a1f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-ba696967-fdf0-4475-96ac-38916e8f7a1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:45.815148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:45.816045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:45.816940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:45.816940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:45.816940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:45.816940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:45.816940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:45.816940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:45.816940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:45.816940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:45.816940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 330/673] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:45.817421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba696967-fdf0-4475-96ac-38916e8f7a1f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.823808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7011815-b852-4614-99fb-6b5f9447e8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:45.825237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7011815-b852-4614-99fb-6b5f9447e8da None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:45.825742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7011815-b852-4614-99fb-6b5f9447e8da None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:45.826231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7011815-b852-4614-99fb-6b5f9447e8da None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:45.827060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7011815-b852-4614-99fb-6b5f9447e8da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:45.827632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7011815-b852-4614-99fb-6b5f9447e8da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:45.829186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7011815-b852-4614-99fb-6b5f9447e8da None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:45.843554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 344/674] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:45.845171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7011815-b852-4614-99fb-6b5f9447e8da None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.911631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:45.925173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba696967-fdf0-4475-96ac-38916e8f7a1f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.959566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693b750-3bd7-4868-acd0-6755f7208786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:45.960324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693b750-3bd7-4868-acd0-6755f7208786 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:45.960608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693b750-3bd7-4868-acd0-6755f7208786 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:45.960707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7011815-b852-4614-99fb-6b5f9447e8da None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:45.960887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693b750-3bd7-4868-acd0-6755f7208786 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:45.961609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693b750-3bd7-4868-acd0-6755f7208786 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:45.961921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693b750-3bd7-4868-acd0-6755f7208786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:45.963364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693b750-3bd7-4868-acd0-6755f7208786 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:45.972588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 331/675] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:45.973432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693b750-3bd7-4868-acd0-6755f7208786 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:45.997501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:46.007476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47044aa-4e68-4077-94f8-a675e2503b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:46.008968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47044aa-4e68-4077-94f8-a675e2503b09 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.009417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47044aa-4e68-4077-94f8-a675e2503b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.010139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47044aa-4e68-4077-94f8-a675e2503b09 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.011414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47044aa-4e68-4077-94f8-a675e2503b09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:46.012637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47044aa-4e68-4077-94f8-a675e2503b09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:46.014554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47044aa-4e68-4077-94f8-a675e2503b09 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:46.037376 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 345/676] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:46.038926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47044aa-4e68-4077-94f8-a675e2503b09 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:46.087000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693b750-3bd7-4868-acd0-6755f7208786 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:46.139092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cfdf0-bc13-482a-8507-332de9f9bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:46.140471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cfdf0-bc13-482a-8507-332de9f9bbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.141150 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cfdf0-bc13-482a-8507-332de9f9bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.141577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cfdf0-bc13-482a-8507-332de9f9bbe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.151363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cfdf0-bc13-482a-8507-332de9f9bbe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:46.154422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cfdf0-bc13-482a-8507-332de9f9bbe8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:46.159551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47044aa-4e68-4077-94f8-a675e2503b09 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-873cfdf0-bc13-482a-8507-332de9f9bbe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:46.170254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:46.171332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:46.172432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:46.172432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:46.172432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:46.172432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:46.172432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:46.172432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:46.172432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:46.172432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:46.179666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 332/677] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:46.179666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cfdf0-bc13-482a-8507-332de9f9bbe8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:46.193030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:46.212489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0243a-b161-4c94-a0e7-3b16910896f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:46.213872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0243a-b161-4c94-a0e7-3b16910896f4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.214184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0243a-b161-4c94-a0e7-3b16910896f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.214460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0243a-b161-4c94-a0e7-3b16910896f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.228460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0243a-b161-4c94-a0e7-3b16910896f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:46.229908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0243a-b161-4c94-a0e7-3b16910896f4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d5d0243a-b161-4c94-a0e7-3b16910896f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:46.245834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:46.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:46.248233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:46.248233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:46.248233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:46.252199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 346/678] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:46.253106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0243a-b161-4c94-a0e7-3b16910896f4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:46.307738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:46.318074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cfdf0-bc13-482a-8507-332de9f9bbe8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:46.367741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378ccbd-0c60-4fdc-883a-f62d2a0bb7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:46.368862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378ccbd-0c60-4fdc-883a-f62d2a0bb7ee None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.369295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378ccbd-0c60-4fdc-883a-f62d2a0bb7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.369993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378ccbd-0c60-4fdc-883a-f62d2a0bb7ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.371825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378ccbd-0c60-4fdc-883a-f62d2a0bb7ee None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=c07403facfaf4c71ab5324af6c44f52a) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:46.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378ccbd-0c60-4fdc-883a-f62d2a0bb7ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:46.374336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378ccbd-0c60-4fdc-883a-f62d2a0bb7ee None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:46.374530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0243a-b161-4c94-a0e7-3b16910896f4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:46.383057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 333/679] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:46 2022] GET /identity/v3/role_assignments?user.id=c07403facfaf4c71ab5324af6c44f52a&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:46.384253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378ccbd-0c60-4fdc-883a-f62d2a0bb7ee None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:46.462592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d48d4-d769-4aad-84e5-e6943d2bc3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:46.465440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d48d4-d769-4aad-84e5-e6943d2bc3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.465883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d48d4-d769-4aad-84e5-e6943d2bc3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.466340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d48d4-d769-4aad-84e5-e6943d2bc3a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.467428 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d48d4-d769-4aad-84e5-e6943d2bc3a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:46.468842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d48d4-d769-4aad-84e5-e6943d2bc3a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:46.473203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d48d4-d769-4aad-84e5-e6943d2bc3a7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:46.489265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 347/680] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:46.497109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d48d4-d769-4aad-84e5-e6943d2bc3a7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:46.525029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:46.561631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378ccbd-0c60-4fdc-883a-f62d2a0bb7ee None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:46.610322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc52f0-30df-4432-bcdc-b9ff7657679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:46.611204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc52f0-30df-4432-bcdc-b9ff7657679b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.611689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc52f0-30df-4432-bcdc-b9ff7657679b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.612136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc52f0-30df-4432-bcdc-b9ff7657679b None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.613292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc52f0-30df-4432-bcdc-b9ff7657679b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:46.613752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc52f0-30df-4432-bcdc-b9ff7657679b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:46.615345 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc52f0-30df-4432-bcdc-b9ff7657679b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:46.629477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 334/681] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:46.631059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08603501-9daa-4473-b9c0-110beb26883f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.631479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08603501-9daa-4473-b9c0-110beb26883f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.631908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08603501-9daa-4473-b9c0-110beb26883f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.633559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 335/682] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:46.634755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08603501-9daa-4473-b9c0-110beb26883f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:46.653870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d48d4-d769-4aad-84e5-e6943d2bc3a7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:46.660353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:46.702035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dd0a4-faa8-4cb6-a302-e500c063b180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:46.702244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dd0a4-faa8-4cb6-a302-e500c063b180 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.702786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dd0a4-faa8-4cb6-a302-e500c063b180 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.702786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dd0a4-faa8-4cb6-a302-e500c063b180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.704633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dd0a4-faa8-4cb6-a302-e500c063b180 None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=63a04998aafd40cc941283700898c92f) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:46.704633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dd0a4-faa8-4cb6-a302-e500c063b180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:46.705103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dd0a4-faa8-4cb6-a302-e500c063b180 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:46.714143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 348/683] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:46 2022] GET /identity/v3/role_assignments?user.id=63a04998aafd40cc941283700898c92f&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:46.715277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3cedb-3702-47d2-bb74-b785532aaeac None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.715572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3cedb-3702-47d2-bb74-b785532aaeac None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.715850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3cedb-3702-47d2-bb74-b785532aaeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.752166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a8c3cedb-3702-47d2-bb74-b785532aaeac None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:46.754384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a8c3cedb-3702-47d2-bb74-b785532aaeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:46.759523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08603501-9daa-4473-b9c0-110beb26883f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:46.801039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7a5b1-412b-4071-8c03-a768d64887b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:46.801732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7a5b1-412b-4071-8c03-a768d64887b0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:46.802043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7a5b1-412b-4071-8c03-a768d64887b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:46.802316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7a5b1-412b-4071-8c03-a768d64887b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:46.805675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:46.813772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7a5b1-412b-4071-8c03-a768d64887b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:46.815910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7a5b1-412b-4071-8c03-a768d64887b0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3da7a5b1-412b-4071-8c03-a768d64887b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:46.830045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:46.831269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:14:46.832440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:46.833988 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 336/684] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:46.839950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7a5b1-412b-4071-8c03-a768d64887b0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:46.858341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 349/685] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:46.995640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7a5b1-412b-4071-8c03-a768d64887b0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:47.046265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9182c9-abdf-4b63-b148-2627de1ac1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:47.046890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9182c9-abdf-4b63-b148-2627de1ac1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:47.047169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9182c9-abdf-4b63-b148-2627de1ac1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:47.047440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9182c9-abdf-4b63-b148-2627de1ac1b0 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:47.048181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9182c9-abdf-4b63-b148-2627de1ac1b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:47.048483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9182c9-abdf-4b63-b148-2627de1ac1b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:47.049991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9182c9-abdf-4b63-b148-2627de1ac1b0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:47.076330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 337/686] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Sep 21 05:14:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:47.084525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3cedb-3702-47d2-bb74-b785532aaeac None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:47.178171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9182c9-abdf-4b63-b148-2627de1ac1b0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:47.205847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:47.211052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3cedb-3702-47d2-bb74-b785532aaeac None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:47.256152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27934d-c09c-48c0-858e-c1849353c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:47.256653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27934d-c09c-48c0-858e-c1849353c79c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:47.257055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27934d-c09c-48c0-858e-c1849353c79c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:47.257333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27934d-c09c-48c0-858e-c1849353c79c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:47.282317 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:47.314048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9182c9-abdf-4b63-b148-2627de1ac1b0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:47.355369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27934d-c09c-48c0-858e-c1849353c79c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:47.357655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27934d-c09c-48c0-858e-c1849353c79c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:47.360108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00da8f-132b-492b-b370-3ec15999dc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:47.360754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00da8f-132b-492b-b370-3ec15999dc2f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:47.362048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00da8f-132b-492b-b370-3ec15999dc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:47.362342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00da8f-132b-492b-b370-3ec15999dc2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:47.376104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00da8f-132b-492b-b370-3ec15999dc2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:47.378446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00da8f-132b-492b-b370-3ec15999dc2f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3d00da8f-132b-492b-b370-3ec15999dc2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:47.398601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:47.399775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:47.400832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:47.400832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:47.400832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:47.400832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 338/687] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:47.406480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00da8f-132b-492b-b370-3ec15999dc2f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:47.472527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:47.478085 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 350/688] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:47.484681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27934d-c09c-48c0-858e-c1849353c79c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:47.521811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00da8f-132b-492b-b370-3ec15999dc2f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:47.538512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:47.562468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e8cb3-aa7a-465f-a045-f50535d61c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:47.563102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e8cb3-aa7a-465f-a045-f50535d61c38 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:47.563519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e8cb3-aa7a-465f-a045-f50535d61c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:47.563850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e8cb3-aa7a-465f-a045-f50535d61c38 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:47.564432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e8cb3-aa7a-465f-a045-f50535d61c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:47.564737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e8cb3-aa7a-465f-a045-f50535d61c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:47.566151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e8cb3-aa7a-465f-a045-f50535d61c38 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:47.579064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 339/689] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:47.594952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e8cb3-aa7a-465f-a045-f50535d61c38 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:47.613163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27934d-c09c-48c0-858e-c1849353c79c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:47.655932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5a5c9-cc28-437e-b547-a52e1a90c089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:47.657101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5a5c9-cc28-437e-b547-a52e1a90c089 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:47.657911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5a5c9-cc28-437e-b547-a52e1a90c089 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:47.658711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5a5c9-cc28-437e-b547-a52e1a90c089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:47.671172 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5a5c9-cc28-437e-b547-a52e1a90c089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:47.673004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5a5c9-cc28-437e-b547-a52e1a90c089 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-86c5a5c9-cc28-437e-b547-a52e1a90c089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:47.690846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:47.691837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:47.692694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:47.692694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:47.692694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:47.695538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 351/690] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:47.701812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4c4b0-98d8-4248-9a39-46ccc0d24ef5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:47.702686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4c4b0-98d8-4248-9a39-46ccc0d24ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:47.702977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4c4b0-98d8-4248-9a39-46ccc0d24ef5 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:47.704464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 352/691] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:47.708430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4c4b0-98d8-4248-9a39-46ccc0d24ef5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:47.711616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e8cb3-aa7a-465f-a045-f50535d61c38 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:47.748780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d01a5-306a-4417-9212-064c2cbe8748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:47.749437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d01a5-306a-4417-9212-064c2cbe8748 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:47.749721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d01a5-306a-4417-9212-064c2cbe8748 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:47.750017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d01a5-306a-4417-9212-064c2cbe8748 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:47.750770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d01a5-306a-4417-9212-064c2cbe8748 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:47.752256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d01a5-306a-4417-9212-064c2cbe8748 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:47.774562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:47.781973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 340/692] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:14:47 2022] POST /identity/v3/users => generated 254 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:14:47.784230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93504ff-37ac-479a-ab61-ebdaeefd0eec None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:47.784621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93504ff-37ac-479a-ab61-ebdaeefd0eec None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:47.785110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93504ff-37ac-479a-ab61-ebdaeefd0eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:47.812018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e93504ff-37ac-479a-ab61-ebdaeefd0eec None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:47.813256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e93504ff-37ac-479a-ab61-ebdaeefd0eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:47.830904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:47.843354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4c4b0-98d8-4248-9a39-46ccc0d24ef5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:47.887663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236679a-7292-48d1-923a-c6a89c98ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:47.888868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236679a-7292-48d1-923a-c6a89c98ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:47.889475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236679a-7292-48d1-923a-c6a89c98ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:47.889815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236679a-7292-48d1-923a-c6a89c98ccd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:47.890587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236679a-7292-48d1-923a-c6a89c98ccd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:47.890965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236679a-7292-48d1-923a-c6a89c98ccd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:47.892565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236679a-7292-48d1-923a-c6a89c98ccd3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:47.902901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 341/693] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:47.916207 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 353/694] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:47.923373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93504ff-37ac-479a-ab61-ebdaeefd0eec None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:48.060621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93504ff-37ac-479a-ab61-ebdaeefd0eec None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:48.102799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b5c96-35c1-45b7-8fee-6c623019453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:48.103720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5c96-35c1-45b7-8fee-6c623019453d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:48.104200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5c96-35c1-45b7-8fee-6c623019453d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:48.104655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5c96-35c1-45b7-8fee-6c623019453d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:48.112310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b5c96-35c1-45b7-8fee-6c623019453d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:48.114234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b5c96-35c1-45b7-8fee-6c623019453d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-9d3b5c96-35c1-45b7-8fee-6c623019453d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:48.126696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:48.127617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:48.128520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:48.128520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:48.128520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:48.128520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:48.128520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:48.128520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:48.128520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:48.128520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:48.130622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 342/695] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:48.136363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236679a-7292-48d1-923a-c6a89c98ccd3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:48.252654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236679a-7292-48d1-923a-c6a89c98ccd3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:48.264571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:48.267999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b5c96-35c1-45b7-8fee-6c623019453d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:48.284115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:48.303330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67e6a2-5b40-454f-a856-e0f716e081a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:48.303946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67e6a2-5b40-454f-a856-e0f716e081a1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:48.304226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67e6a2-5b40-454f-a856-e0f716e081a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:48.304496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67e6a2-5b40-454f-a856-e0f716e081a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:48.305185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67e6a2-5b40-454f-a856-e0f716e081a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:48.305493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67e6a2-5b40-454f-a856-e0f716e081a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:48.307042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67e6a2-5b40-454f-a856-e0f716e081a1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:48.319988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 354/696] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:48.325288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67e6a2-5b40-454f-a856-e0f716e081a1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:48.388388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b5c96-35c1-45b7-8fee-6c623019453d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:48.424965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12e1cb-5784-4963-873b-1f44e9f01810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:48.425959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12e1cb-5784-4963-873b-1f44e9f01810 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:48.426452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12e1cb-5784-4963-873b-1f44e9f01810 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:48.426997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12e1cb-5784-4963-873b-1f44e9f01810 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:48.433031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67e6a2-5b40-454f-a856-e0f716e081a1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:48.438828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12e1cb-5784-4963-873b-1f44e9f01810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-bf12e1cb-5784-4963-873b-1f44e9f01810 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:48.445287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:48.446236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:48.447094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:48.449894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 343/697] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:48.455737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12e1cb-5784-4963-873b-1f44e9f01810 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:48.476933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362f42d-687c-4994-a60b-40040d6aa9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:48.478321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362f42d-687c-4994-a60b-40040d6aa9cf None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:48.479075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362f42d-687c-4994-a60b-40040d6aa9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:48.479563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362f42d-687c-4994-a60b-40040d6aa9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:48.481202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362f42d-687c-4994-a60b-40040d6aa9cf None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=ee0fe35d9e304ee495998c93c6909fa8) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:48.482048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362f42d-687c-4994-a60b-40040d6aa9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:48.483681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362f42d-687c-4994-a60b-40040d6aa9cf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:48.491987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 355/698] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:48 2022] GET /identity/v3/role_assignments?user.id=ee0fe35d9e304ee495998c93c6909fa8&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:48.570694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:48.641873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12e1cb-5784-4963-873b-1f44e9f01810 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:48.696005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb416d9d-23af-4d1b-8d94-ad73e3600dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:48.696697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb416d9d-23af-4d1b-8d94-ad73e3600dbb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:48.696981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb416d9d-23af-4d1b-8d94-ad73e3600dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:48.697254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb416d9d-23af-4d1b-8d94-ad73e3600dbb None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:48.698004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb416d9d-23af-4d1b-8d94-ad73e3600dbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:48.698311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb416d9d-23af-4d1b-8d94-ad73e3600dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:48.699761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb416d9d-23af-4d1b-8d94-ad73e3600dbb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:48.711300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 344/699] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:48.719046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362f42d-687c-4994-a60b-40040d6aa9cf None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:48.758529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:48.833562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362f42d-687c-4994-a60b-40040d6aa9cf None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:48.874719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad713b71-120c-42e0-8f17-b6e541109ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:48.875355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad713b71-120c-42e0-8f17-b6e541109ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:48.875636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad713b71-120c-42e0-8f17-b6e541109ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:48.875905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad713b71-120c-42e0-8f17-b6e541109ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:48.888525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad713b71-120c-42e0-8f17-b6e541109ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:48.890001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad713b71-120c-42e0-8f17-b6e541109ce4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ad713b71-120c-42e0-8f17-b6e541109ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:48.905995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:48.906930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:48.907814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:48.907814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:48.907814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:48.910508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 356/700] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:48.915876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb416d9d-23af-4d1b-8d94-ad73e3600dbb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:49.029583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb416d9d-23af-4d1b-8d94-ad73e3600dbb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:49.063793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:49.070106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b3ebc-b2cb-4783-95a1-c307db0a4887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:49.070931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b3ebc-b2cb-4783-95a1-c307db0a4887 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:49.071226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b3ebc-b2cb-4783-95a1-c307db0a4887 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:49.071503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b3ebc-b2cb-4783-95a1-c307db0a4887 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:49.072143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b3ebc-b2cb-4783-95a1-c307db0a4887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:49.072449 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b3ebc-b2cb-4783-95a1-c307db0a4887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:49.073863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b3ebc-b2cb-4783-95a1-c307db0a4887 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:49.092997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 345/701] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:49.094338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad713b71-120c-42e0-8f17-b6e541109ce4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:49.204786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:49.222477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad713b71-120c-42e0-8f17-b6e541109ce4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:49.255545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0cf0c-c2a6-44db-8453-03bf63d52cef None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:49.256136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0cf0c-c2a6-44db-8453-03bf63d52cef None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:49.256654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0cf0c-c2a6-44db-8453-03bf63d52cef None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:49.258415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 346/702] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:49.264894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef87fa-d280-433d-b6ca-d84f12e126b0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:49.265259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef87fa-d280-433d-b6ca-d84f12e126b0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:49.265599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef87fa-d280-433d-b6ca-d84f12e126b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:49.266425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1334d-7c1e-4967-b3c5-3f96ad9036c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:49.267586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1334d-7c1e-4967-b3c5-3f96ad9036c2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:49.268094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1334d-7c1e-4967-b3c5-3f96ad9036c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:49.268594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1334d-7c1e-4967-b3c5-3f96ad9036c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:49.277621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1334d-7c1e-4967-b3c5-3f96ad9036c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:49.279400 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1334d-7c1e-4967-b3c5-3f96ad9036c2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c1f1334d-7c1e-4967-b3c5-3f96ad9036c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:49.291229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:49.292323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:49.293300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:49.300176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 357/703] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Sep 21 05:14:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:49.302573 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1334d-7c1e-4967-b3c5-3f96ad9036c2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:49.311135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8fef87fa-d280-433d-b6ca-d84f12e126b0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:49.312345 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8fef87fa-d280-433d-b6ca-d84f12e126b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:49.385246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 347/704] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 05:14:49.469616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1334d-7c1e-4967-b3c5-3f96ad9036c2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:49.517357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8225f9-f872-4cf8-8e37-5516ccf0db79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:49.517987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8225f9-f872-4cf8-8e37-5516ccf0db79 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:49.518274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8225f9-f872-4cf8-8e37-5516ccf0db79 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:49.518548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8225f9-f872-4cf8-8e37-5516ccf0db79 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:49.519269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8225f9-f872-4cf8-8e37-5516ccf0db79 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:49.519571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8225f9-f872-4cf8-8e37-5516ccf0db79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:49.520997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8225f9-f872-4cf8-8e37-5516ccf0db79 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:49.538982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 358/705] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Sep 21 05:14:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:49.544645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef87fa-d280-433d-b6ca-d84f12e126b0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:49.678646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef87fa-d280-433d-b6ca-d84f12e126b0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:49.703156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:49.714700 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8225f9-f872-4cf8-8e37-5516ccf0db79 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:49.719142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:49.720378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28566959-baa4-4a99-9193-7bbfabb54e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:49.720993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28566959-baa4-4a99-9193-7bbfabb54e3c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:49.721533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28566959-baa4-4a99-9193-7bbfabb54e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:49.721811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28566959-baa4-4a99-9193-7bbfabb54e3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:49.884701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28566959-baa4-4a99-9193-7bbfabb54e3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:49.886600 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28566959-baa4-4a99-9193-7bbfabb54e3c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:49.898767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8225f9-f872-4cf8-8e37-5516ccf0db79 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:49.935758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449f4e3-c1a3-4193-986b-dd9c56097c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:49.936313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449f4e3-c1a3-4193-986b-dd9c56097c84 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:49.936592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449f4e3-c1a3-4193-986b-dd9c56097c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:49.936866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449f4e3-c1a3-4193-986b-dd9c56097c84 None admin] PATH_INFO: `/v3/services/network` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:49.937440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449f4e3-c1a3-4193-986b-dd9c56097c84 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:49.941548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449f4e3-c1a3-4193-986b-dd9c56097c84 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d449f4e3-c1a3-4193-986b-dd9c56097c84 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:49.953795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:49.954830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:49.956072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 359/706] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:49 2022] GET /identity/v3/services/network => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:49.956852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aa908-fe05-405a-afd8-af53bfb111fc None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:49.956852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aa908-fe05-405a-afd8-af53bfb111fc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:49.956987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aa908-fe05-405a-afd8-af53bfb111fc None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:49.967698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 360/707] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:49.967698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078aa908-fe05-405a-afd8-af53bfb111fc None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:50.031313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 348/708] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:50.032622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426bb95-3af3-4668-810d-61e56cf7ee4f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:50.032835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426bb95-3af3-4668-810d-61e56cf7ee4f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:50.033109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426bb95-3af3-4668-810d-61e56cf7ee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:50.050839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:50.054041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7426bb95-3af3-4668-810d-61e56cf7ee4f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:50.056732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7426bb95-3af3-4668-810d-61e56cf7ee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:50.079922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:50.088804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078aa908-fe05-405a-afd8-af53bfb111fc None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:50.134480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a5514-d192-4daa-b98c-cebdb31ec2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:50.135180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a5514-d192-4daa-b98c-cebdb31ec2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:50.135521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a5514-d192-4daa-b98c-cebdb31ec2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:50.136024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a5514-d192-4daa-b98c-cebdb31ec2d4 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:50.136746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a5514-d192-4daa-b98c-cebdb31ec2d4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:50.137346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a5514-d192-4daa-b98c-cebdb31ec2d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:50.139027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a5514-d192-4daa-b98c-cebdb31ec2d4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:50.146039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 349/710] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:50.147232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426bb95-3af3-4668-810d-61e56cf7ee4f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:50.149982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 361/710] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:14:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:50.149982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a5514-d192-4daa-b98c-cebdb31ec2d4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:50.289956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a5514-d192-4daa-b98c-cebdb31ec2d4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:50.319139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426bb95-3af3-4668-810d-61e56cf7ee4f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:50.352102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:50.361767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd75d-80c3-41bd-b566-b9383cea4222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:50.363295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd75d-80c3-41bd-b566-b9383cea4222 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:50.364759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd75d-80c3-41bd-b566-b9383cea4222 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:50.365402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd75d-80c3-41bd-b566-b9383cea4222 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:50.367338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd75d-80c3-41bd-b566-b9383cea4222 None admin] RBAC: Adding query filter params (type=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:50.368714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd75d-80c3-41bd-b566-b9383cea4222 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:50.370329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd75d-80c3-41bd-b566-b9383cea4222 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:50.384163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 362/711] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:14:50 2022] GET /identity/v3/services?type=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:50.388048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2a48b-dcc1-4796-92da-b811ed6dc8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:50.389054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2a48b-dcc1-4796-92da-b811ed6dc8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:50.389535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2a48b-dcc1-4796-92da-b811ed6dc8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:50.390064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2a48b-dcc1-4796-92da-b811ed6dc8e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:50.401854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2a48b-dcc1-4796-92da-b811ed6dc8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:50.404190 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2a48b-dcc1-4796-92da-b811ed6dc8e5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-fbd2a48b-dcc1-4796-92da-b811ed6dc8e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:50.421390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:50.422641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:50.423766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:50.423766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:50.423766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:50.429864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 350/712] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:50.431480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd75d-80c3-41bd-b566-b9383cea4222 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:50.571071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd75d-80c3-41bd-b566-b9383cea4222 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:50.600488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2a48b-dcc1-4796-92da-b811ed6dc8e5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:50.618291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:50.658092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b439b8b-19ec-4726-8ca9-9c5dea1dcc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:50.658727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b439b8b-19ec-4726-8ca9-9c5dea1dcc92 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:50.659005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b439b8b-19ec-4726-8ca9-9c5dea1dcc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:50.659273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b439b8b-19ec-4726-8ca9-9c5dea1dcc92 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:50.659919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b439b8b-19ec-4726-8ca9-9c5dea1dcc92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:50.660252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b439b8b-19ec-4726-8ca9-9c5dea1dcc92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:50.662322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b439b8b-19ec-4726-8ca9-9c5dea1dcc92 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:50.680582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 363/713] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:50.686261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b439b8b-19ec-4726-8ca9-9c5dea1dcc92 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:50.754357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2a48b-dcc1-4796-92da-b811ed6dc8e5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:50.784687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:50.807560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b8d56-e6ed-402b-bdf3-a3c94f99adf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:50.808197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8d56-e6ed-402b-bdf3-a3c94f99adf7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:50.808469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8d56-e6ed-402b-bdf3-a3c94f99adf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:50.809334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8d56-e6ed-402b-bdf3-a3c94f99adf7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:50.809929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b8d56-e6ed-402b-bdf3-a3c94f99adf7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:50.811495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b8d56-e6ed-402b-bdf3-a3c94f99adf7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-548b8d56-e6ed-402b-bdf3-a3c94f99adf7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:50.824925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:50.825916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 05:14:50.826826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:50.829730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 351/714] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Sep 21 05:14:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:50.831327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388eb216-6c6a-40fc-9ecf-eb65d8910634 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:50.831836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388eb216-6c6a-40fc-9ecf-eb65d8910634 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:50.832330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388eb216-6c6a-40fc-9ecf-eb65d8910634 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:50.838134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 352/715] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:50.839152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388eb216-6c6a-40fc-9ecf-eb65d8910634 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:50.844035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b439b8b-19ec-4726-8ca9-9c5dea1dcc92 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:50.883058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e12c7-a6f7-4a65-ac05-a02e31f5ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:50.883757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e12c7-a6f7-4a65-ac05-a02e31f5ee97 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:50.884090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e12c7-a6f7-4a65-ac05-a02e31f5ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:50.884373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e12c7-a6f7-4a65-ac05-a02e31f5ee97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:50.892211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e12c7-a6f7-4a65-ac05-a02e31f5ee97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:50.893990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e12c7-a6f7-4a65-ac05-a02e31f5ee97 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ba1e12c7-a6f7-4a65-ac05-a02e31f5ee97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:50.907940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:50.908873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:50.910145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:50.910145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:50.910145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:50.910145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:50.910145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:50.910145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:50.910145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:50.910145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:50.910145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 364/716] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:50.914510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ee5e7-c108-44e2-8a50-1225db5432aa None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:50.915720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ee5e7-c108-44e2-8a50-1225db5432aa None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:50.916822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ee5e7-c108-44e2-8a50-1225db5432aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:50.925057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:50.940386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8c2ee5e7-c108-44e2-8a50-1225db5432aa None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:50.941996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8c2ee5e7-c108-44e2-8a50-1225db5432aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:50.967202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388eb216-6c6a-40fc-9ecf-eb65d8910634 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:51.012630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe4e53-5fd1-4661-ab83-ff716e2cab61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:51.013235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe4e53-5fd1-4661-ab83-ff716e2cab61 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:51.013514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe4e53-5fd1-4661-ab83-ff716e2cab61 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:51.013785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe4e53-5fd1-4661-ab83-ff716e2cab61 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:51.014438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe4e53-5fd1-4661-ab83-ff716e2cab61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:51.014743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe4e53-5fd1-4661-ab83-ff716e2cab61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:51.016110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe4e53-5fd1-4661-ab83-ff716e2cab61 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:51.026158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 365/718] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:51.026158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ee5e7-c108-44e2-8a50-1225db5432aa None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:51.026540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 353/718] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Sep 21 05:14:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:51.030112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe4e53-5fd1-4661-ab83-ff716e2cab61 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:51.079208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:51.203963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ee5e7-c108-44e2-8a50-1225db5432aa None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:51.205428 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe4e53-5fd1-4661-ab83-ff716e2cab61 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:51.240556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:51.247859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6958842-f84b-44d0-b6ad-e25759c22c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:51.248872 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6958842-f84b-44d0-b6ad-e25759c22c53 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:51.249303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6958842-f84b-44d0-b6ad-e25759c22c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:51.249879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6958842-f84b-44d0-b6ad-e25759c22c53 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:51.251234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6958842-f84b-44d0-b6ad-e25759c22c53 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:51.251692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6958842-f84b-44d0-b6ad-e25759c22c53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:51.253206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6958842-f84b-44d0-b6ad-e25759c22c53 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:51.260468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 354/719] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Sep 21 05:14:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:51.273735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c016fdf-90d7-4418-bb74-af4118f63963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:51.274639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c016fdf-90d7-4418-bb74-af4118f63963 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:51.275069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c016fdf-90d7-4418-bb74-af4118f63963 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:51.275537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c016fdf-90d7-4418-bb74-af4118f63963 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:51.276416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c016fdf-90d7-4418-bb74-af4118f63963 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:51.276885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c016fdf-90d7-4418-bb74-af4118f63963 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:51.278538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c016fdf-90d7-4418-bb74-af4118f63963 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:51.293557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 366/720] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:51.303189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6958842-f84b-44d0-b6ad-e25759c22c53 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:51.318414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c016fdf-90d7-4418-bb74-af4118f63963 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:51.448927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c016fdf-90d7-4418-bb74-af4118f63963 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:51.449167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6958842-f84b-44d0-b6ad-e25759c22c53 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:51.459558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:51.492652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdb6ac-0b5a-4c99-b9ff-5152cfb0a2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:51.493262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdb6ac-0b5a-4c99-b9ff-5152cfb0a2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:51.493543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdb6ac-0b5a-4c99-b9ff-5152cfb0a2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:51.493815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdb6ac-0b5a-4c99-b9ff-5152cfb0a2c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:51.494568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdb6ac-0b5a-4c99-b9ff-5152cfb0a2c0 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d9168d8bbf8b43299a7748c9faa099b1) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:51.494882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdb6ac-0b5a-4c99-b9ff-5152cfb0a2c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:51.496365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdb6ac-0b5a-4c99-b9ff-5152cfb0a2c0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:51.497240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c96c46-de1c-4203-af39-6248ee74b91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:51.497871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c96c46-de1c-4203-af39-6248ee74b91c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:51.498237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c96c46-de1c-4203-af39-6248ee74b91c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:51.498445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c96c46-de1c-4203-af39-6248ee74b91c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:51.502101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 355/721] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:51 2022] GET /identity/v3/role_assignments?user.id=d9168d8bbf8b43299a7748c9faa099b1&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:51.505043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b822d6-1672-4e11-bfb4-2edac29d1400 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:51.505043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b822d6-1672-4e11-bfb4-2edac29d1400 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:51.505043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b822d6-1672-4e11-bfb4-2edac29d1400 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:51.505043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 356/722] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:51.510674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5225cf-4140-49b1-85d9-f7f20fea3894 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:51.510989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5225cf-4140-49b1-85d9-f7f20fea3894 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:51.511230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5225cf-4140-49b1-85d9-f7f20fea3894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:51.512812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c96c46-de1c-4203-af39-6248ee74b91c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c7c96c46-de1c-4203-af39-6248ee74b91c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:51.518449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:51.519556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:51.520423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 367/723] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:51.525991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c96c46-de1c-4203-af39-6248ee74b91c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:51.544024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-dc5225cf-4140-49b1-85d9-f7f20fea3894 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:51.546493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dc5225cf-4140-49b1-85d9-f7f20fea3894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:51.658719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:51.673577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 357/724] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:51.712586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c96c46-de1c-4203-af39-6248ee74b91c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:51.758435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50439-a938-460e-bbf1-a87fc68863fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:51.759041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50439-a938-460e-bbf1-a87fc68863fd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:51.759316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50439-a938-460e-bbf1-a87fc68863fd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:51.759590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50439-a938-460e-bbf1-a87fc68863fd None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:51.760237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50439-a938-460e-bbf1-a87fc68863fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:51.760535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50439-a938-460e-bbf1-a87fc68863fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:51.761927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50439-a938-460e-bbf1-a87fc68863fd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:51.770889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 368/725] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:51.776318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5225cf-4140-49b1-85d9-f7f20fea3894 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:51.923763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5225cf-4140-49b1-85d9-f7f20fea3894 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:51.970162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbb975-650d-4c6f-bafe-d42102e16cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:51.970717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbb975-650d-4c6f-bafe-d42102e16cca None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:51.970993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbb975-650d-4c6f-bafe-d42102e16cca None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:51.971264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbb975-650d-4c6f-bafe-d42102e16cca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:51.984013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbb975-650d-4c6f-bafe-d42102e16cca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:51.985479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbb975-650d-4c6f-bafe-d42102e16cca None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-2adbb975-650d-4c6f-bafe-d42102e16cca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:52.004313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:52.005255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:52.006184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:52.006184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:52.006184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:52.006184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 358/726] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:52.011281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50439-a938-460e-bbf1-a87fc68863fd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:52.062966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:52.085121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbb975-650d-4c6f-bafe-d42102e16cca None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:52.142241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:52.187377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50439-a938-460e-bbf1-a87fc68863fd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:52.257750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbb975-650d-4c6f-bafe-d42102e16cca None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:52.268535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3e00b-ea4d-4941-942e-69209572065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:52.269155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3e00b-ea4d-4941-942e-69209572065e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:52.269437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3e00b-ea4d-4941-942e-69209572065e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:52.269710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3e00b-ea4d-4941-942e-69209572065e None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:52.270379 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3e00b-ea4d-4941-942e-69209572065e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:52.270683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3e00b-ea4d-4941-942e-69209572065e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:52.272037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3e00b-ea4d-4941-942e-69209572065e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:52.289610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 369/727] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:52.291146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3e00b-ea4d-4941-942e-69209572065e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:52.320668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180fbf2-7e77-440c-8c5f-5a92ca6c8b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:52.324909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180fbf2-7e77-440c-8c5f-5a92ca6c8b92 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:52.325191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180fbf2-7e77-440c-8c5f-5a92ca6c8b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:52.325464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180fbf2-7e77-440c-8c5f-5a92ca6c8b92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:52.350696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180fbf2-7e77-440c-8c5f-5a92ca6c8b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-8180fbf2-7e77-440c-8c5f-5a92ca6c8b92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:52.364083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:52.365101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:52.365923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 359/728] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:52.372059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180fbf2-7e77-440c-8c5f-5a92ca6c8b92 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:52.478964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:52.505848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180fbf2-7e77-440c-8c5f-5a92ca6c8b92 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:52.508177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:52.513330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3e00b-ea4d-4941-942e-69209572065e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:52.549054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1c62b-9612-4151-94f4-dcae91b2859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:52.550425 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c62b-9612-4151-94f4-dcae91b2859d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:52.550911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c62b-9612-4151-94f4-dcae91b2859d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:52.551389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c62b-9612-4151-94f4-dcae91b2859d None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:52.552264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1c62b-9612-4151-94f4-dcae91b2859d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:52.552767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1c62b-9612-4151-94f4-dcae91b2859d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:52.554568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1c62b-9612-4151-94f4-dcae91b2859d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:52.565899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 360/729] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:52.571640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1c62b-9612-4151-94f4-dcae91b2859d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:52.597521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496063d-cd05-49e5-b18d-f90e8a36d6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:52.599372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496063d-cd05-49e5-b18d-f90e8a36d6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:52.601190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496063d-cd05-49e5-b18d-f90e8a36d6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:52.601669 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496063d-cd05-49e5-b18d-f90e8a36d6f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:52.618023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496063d-cd05-49e5-b18d-f90e8a36d6f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:52.619474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496063d-cd05-49e5-b18d-f90e8a36d6f1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4496063d-cd05-49e5-b18d-f90e8a36d6f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:52.634274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:52.635277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:52.636501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 370/730] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:14:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:52.644233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496063d-cd05-49e5-b18d-f90e8a36d6f1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:52.757844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496063d-cd05-49e5-b18d-f90e8a36d6f1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:52.796392 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047424cb-f24b-42fe-a406-4c1d0c46ff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:52.796983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047424cb-f24b-42fe-a406-4c1d0c46ff2e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:52.797267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047424cb-f24b-42fe-a406-4c1d0c46ff2e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:52.797539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047424cb-f24b-42fe-a406-4c1d0c46ff2e None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:52.798295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047424cb-f24b-42fe-a406-4c1d0c46ff2e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:52.798599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047424cb-f24b-42fe-a406-4c1d0c46ff2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:52.800030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047424cb-f24b-42fe-a406-4c1d0c46ff2e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:52.820541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 371/731] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Sep 21 05:14:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:52.826810 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047424cb-f24b-42fe-a406-4c1d0c46ff2e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:52.829487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1c62b-9612-4151-94f4-dcae91b2859d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:52.842292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:52.892877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfad57f-17c6-421d-8b0b-1a7be030d51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:52.894144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfad57f-17c6-421d-8b0b-1a7be030d51c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:52.894440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfad57f-17c6-421d-8b0b-1a7be030d51c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:52.894715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfad57f-17c6-421d-8b0b-1a7be030d51c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:52.909608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfad57f-17c6-421d-8b0b-1a7be030d51c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:52.911154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfad57f-17c6-421d-8b0b-1a7be030d51c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-cbfad57f-17c6-421d-8b0b-1a7be030d51c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:52.947393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:52.948378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:52.949282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:52.949282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:52.949282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:52.951442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 361/732] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:52.956494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfad57f-17c6-421d-8b0b-1a7be030d51c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:52.994593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047424cb-f24b-42fe-a406-4c1d0c46ff2e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:53.004626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:53.034995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb1710-6603-4970-994f-01d8cb77a437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:53.035854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb1710-6603-4970-994f-01d8cb77a437 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.036677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb1710-6603-4970-994f-01d8cb77a437 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.037100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb1710-6603-4970-994f-01d8cb77a437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.115877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb1710-6603-4970-994f-01d8cb77a437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:53.117972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb1710-6603-4970-994f-01d8cb77a437 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:53.134252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfad57f-17c6-421d-8b0b-1a7be030d51c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:53.134521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:53.170673 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888edbe-4f80-406f-92d0-73dbbaa0d611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:53.171422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888edbe-4f80-406f-92d0-73dbbaa0d611 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.171802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888edbe-4f80-406f-92d0-73dbbaa0d611 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.172171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888edbe-4f80-406f-92d0-73dbbaa0d611 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.172904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888edbe-4f80-406f-92d0-73dbbaa0d611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:53.173319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888edbe-4f80-406f-92d0-73dbbaa0d611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:53.174841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888edbe-4f80-406f-92d0-73dbbaa0d611 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:53.191543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 362/733] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:53.192958 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdc516-7772-4d81-8e4f-1c873483a0f6 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.193335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdc516-7772-4d81-8e4f-1c873483a0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.193704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdc516-7772-4d81-8e4f-1c873483a0f6 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.199616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 363/734] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:53.200560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdc516-7772-4d81-8e4f-1c873483a0f6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:53.244704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 372/735] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:53.246280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98100-ebb0-4178-ae15-c7fd2be96489 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.246728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98100-ebb0-4178-ae15-c7fd2be96489 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.247181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98100-ebb0-4178-ae15-c7fd2be96489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.268877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7bb98100-ebb0-4178-ae15-c7fd2be96489 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:53.270986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7bb98100-ebb0-4178-ae15-c7fd2be96489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:53.291830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:53.319378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdc516-7772-4d81-8e4f-1c873483a0f6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:53.354431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 373/736] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:53.355406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98100-ebb0-4178-ae15-c7fd2be96489 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:53.367148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c8a7-2d54-4263-801a-7382f8feee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:53.368403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c8a7-2d54-4263-801a-7382f8feee39 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.369146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c8a7-2d54-4263-801a-7382f8feee39 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.369603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c8a7-2d54-4263-801a-7382f8feee39 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.380104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527c8a7-2d54-4263-801a-7382f8feee39 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:53.382598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527c8a7-2d54-4263-801a-7382f8feee39 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:53.392067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-a527c8a7-2d54-4263-801a-7382f8feee39 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:53.398015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:53.398963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:14:53.399882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:53.402956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 364/737] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:53.412211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c8a7-2d54-4263-801a-7382f8feee39 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:53.516848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98100-ebb0-4178-ae15-c7fd2be96489 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:53.580410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3833d64-4dfb-4605-85cc-e5839f10a354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:53.581051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3833d64-4dfb-4605-85cc-e5839f10a354 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.581329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3833d64-4dfb-4605-85cc-e5839f10a354 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.581595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3833d64-4dfb-4605-85cc-e5839f10a354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.601538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3833d64-4dfb-4605-85cc-e5839f10a354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:53.603014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3833d64-4dfb-4605-85cc-e5839f10a354 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:53.610831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c8a7-2d54-4263-801a-7382f8feee39 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f3833d64-4dfb-4605-85cc-e5839f10a354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:53.635596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:53.637098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:53.638012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:53.638012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:53.638012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:53.638354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 374/738] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:53.647911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3833d64-4dfb-4605-85cc-e5839f10a354 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:53.665726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c71a3-bb6f-44df-b653-2a91767a0851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:53.666799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c71a3-bb6f-44df-b653-2a91767a0851 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.667261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c71a3-bb6f-44df-b653-2a91767a0851 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.667745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c71a3-bb6f-44df-b653-2a91767a0851 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.668851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c71a3-bb6f-44df-b653-2a91767a0851 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:53.669476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c71a3-bb6f-44df-b653-2a91767a0851 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:53.670983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c71a3-bb6f-44df-b653-2a91767a0851 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:53.674226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:53.692682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 365/739] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Sep 21 05:14:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:53.698234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c71a3-bb6f-44df-b653-2a91767a0851 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:53.757735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:53.774271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3833d64-4dfb-4605-85cc-e5839f10a354 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:53.813734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6cb4-5023-45b4-8f06-a336aaca9fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:53.814611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6cb4-5023-45b4-8f06-a336aaca9fdd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.815077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6cb4-5023-45b4-8f06-a336aaca9fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.815502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6cb4-5023-45b4-8f06-a336aaca9fdd None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.816041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c71a3-bb6f-44df-b653-2a91767a0851 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:53.816444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6cb4-5023-45b4-8f06-a336aaca9fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:53.816896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6cb4-5023-45b4-8f06-a336aaca9fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:53.818474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6cb4-5023-45b4-8f06-a336aaca9fdd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:53.831334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 375/740] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:53.832221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6cb4-5023-45b4-8f06-a336aaca9fdd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:53.853596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e7c0-2e4d-4b83-abbe-401bd8259b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:53.854336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e7c0-2e4d-4b83-abbe-401bd8259b93 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.854619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e7c0-2e4d-4b83-abbe-401bd8259b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.854892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e7c0-2e4d-4b83-abbe-401bd8259b93 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.855568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4e7c0-2e4d-4b83-abbe-401bd8259b93 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:53.856933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4e7c0-2e4d-4b83-abbe-401bd8259b93 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:53.875135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 366/741] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:14:53 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:14:53.876341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222e2d-e9b8-406e-98b7-5874c632b155 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:53.876645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222e2d-e9b8-406e-98b7-5874c632b155 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:53.876909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222e2d-e9b8-406e-98b7-5874c632b155 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:53.884492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 367/742] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:53.884492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46222e2d-e9b8-406e-98b7-5874c632b155 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:53.972239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6cb4-5023-45b4-8f06-a336aaca9fdd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:53.977824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:54.028846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a81b5-cf8c-49e5-9050-cc387696fefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:54.029475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a81b5-cf8c-49e5-9050-cc387696fefe None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.029806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a81b5-cf8c-49e5-9050-cc387696fefe None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.030186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a81b5-cf8c-49e5-9050-cc387696fefe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.042955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46222e2d-e9b8-406e-98b7-5874c632b155 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:54.047566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:54.085376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f8322-272c-44d1-9cc3-6584bd247cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:54.086143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f8322-272c-44d1-9cc3-6584bd247cad None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.086422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f8322-272c-44d1-9cc3-6584bd247cad None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.086692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f8322-272c-44d1-9cc3-6584bd247cad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.093651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f8322-272c-44d1-9cc3-6584bd247cad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:54.095208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f8322-272c-44d1-9cc3-6584bd247cad None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-4f3f8322-272c-44d1-9cc3-6584bd247cad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:54.108175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:54.109128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:54.110049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:54.110049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:54.110049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:54.110049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:54.110049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:54.110049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:54.110049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:54.110049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:54.110049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 368/743] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:54.110546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0731e-2c43-4ffa-8df9-216fc75bc10b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.111571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0731e-2c43-4ffa-8df9-216fc75bc10b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.111571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0731e-2c43-4ffa-8df9-216fc75bc10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.116525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a81b5-cf8c-49e5-9050-cc387696fefe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:54.118020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a81b5-cf8c-49e5-9050-cc387696fefe None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:54.129509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4ec0731e-2c43-4ffa-8df9-216fc75bc10b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:54.130713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4ec0731e-2c43-4ffa-8df9-216fc75bc10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:54.201445 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 369/744] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:54 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:54.202561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2002e-120a-47ca-9b20-1885811edb11 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.202840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2002e-120a-47ca-9b20-1885811edb11 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.203109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2002e-120a-47ca-9b20-1885811edb11 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.204450 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 370/745] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:54.206672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2002e-120a-47ca-9b20-1885811edb11 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:54.236319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 376/746] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:14:53 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 05:14:54.246655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbad6b5-b639-42c7-9498-cb434a3d68ec None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.246655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbad6b5-b639-42c7-9498-cb434a3d68ec None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.246655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbad6b5-b639-42c7-9498-cb434a3d68ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.258429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8bbad6b5-b639-42c7-9498-cb434a3d68ec None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:54.259858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8bbad6b5-b639-42c7-9498-cb434a3d68ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:54.282971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:54.312856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:54.346977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 377/747] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:54 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 05:14:54.348211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbad6b5-b639-42c7-9498-cb434a3d68ec None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:54.357549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2002e-120a-47ca-9b20-1885811edb11 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:54.434752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe2d57-b530-4120-b504-5c1e5c2e34d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:54.435358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe2d57-b530-4120-b504-5c1e5c2e34d5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.435635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe2d57-b530-4120-b504-5c1e5c2e34d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.435903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe2d57-b530-4120-b504-5c1e5c2e34d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.436587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe2d57-b530-4120-b504-5c1e5c2e34d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:54.436887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe2d57-b530-4120-b504-5c1e5c2e34d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:54.438335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe2d57-b530-4120-b504-5c1e5c2e34d5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:54.472406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 371/748] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:54.478104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe2d57-b530-4120-b504-5c1e5c2e34d5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:54.484211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbad6b5-b639-42c7-9498-cb434a3d68ec None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:54.518916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fae9e-c2fe-4ef4-a79d-76aea39331f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:54.519514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fae9e-c2fe-4ef4-a79d-76aea39331f8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.519734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fae9e-c2fe-4ef4-a79d-76aea39331f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.519903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fae9e-c2fe-4ef4-a79d-76aea39331f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.530405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fae9e-c2fe-4ef4-a79d-76aea39331f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:54.531875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fae9e-c2fe-4ef4-a79d-76aea39331f8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-353fae9e-c2fe-4ef4-a79d-76aea39331f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:54.546759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:54.547739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:54.548599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:54.548599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:54.548599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:54.548599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 378/749] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:54.548599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fae9e-c2fe-4ef4-a79d-76aea39331f8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:54.580291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:54.623512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe2d57-b530-4120-b504-5c1e5c2e34d5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:54.626326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:54.658538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fae9e-c2fe-4ef4-a79d-76aea39331f8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:54.663015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a4d10-08f4-48c0-b98a-a51d689e0de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:54.663688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a4d10-08f4-48c0-b98a-a51d689e0de0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.663975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a4d10-08f4-48c0-b98a-a51d689e0de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.664250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a4d10-08f4-48c0-b98a-a51d689e0de0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.664975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a4d10-08f4-48c0-b98a-a51d689e0de0 None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d2297673f49b4ce8bf40ac5f750d989f) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:54.665286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a4d10-08f4-48c0-b98a-a51d689e0de0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:54.666762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a4d10-08f4-48c0-b98a-a51d689e0de0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:54.673072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 372/750] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:54 2022] GET /identity/v3/role_assignments?user.id=d2297673f49b4ce8bf40ac5f750d989f&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:54.678049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a4d10-08f4-48c0-b98a-a51d689e0de0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:54.703153 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23933a3f-4e87-4714-8cea-c9e0e8473c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:54.703782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23933a3f-4e87-4714-8cea-c9e0e8473c1e None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.704059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23933a3f-4e87-4714-8cea-c9e0e8473c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.704330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23933a3f-4e87-4714-8cea-c9e0e8473c1e None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.704992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23933a3f-4e87-4714-8cea-c9e0e8473c1e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:54.706433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23933a3f-4e87-4714-8cea-c9e0e8473c1e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:54.734031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 379/751] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:14:54 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:14:54.734031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23933a3f-4e87-4714-8cea-c9e0e8473c1e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:54.846364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:54.867089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23933a3f-4e87-4714-8cea-c9e0e8473c1e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:54.875537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a4d10-08f4-48c0-b98a-a51d689e0de0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:54.907948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb253-2863-466e-9234-0063e50cde2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:54.908593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb253-2863-466e-9234-0063e50cde2e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.908871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb253-2863-466e-9234-0063e50cde2e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.909143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb253-2863-466e-9234-0063e50cde2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.921102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ceb253-2863-466e-9234-0063e50cde2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b3ceb253-2863-466e-9234-0063e50cde2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:54.926753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:54.927749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:54.928803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 380/752] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:54.938835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb253-2863-466e-9234-0063e50cde2e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:54.953566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb8690-117c-4580-a32a-61bff7e43009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:54.954233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb8690-117c-4580-a32a-61bff7e43009 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:54.954513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb8690-117c-4580-a32a-61bff7e43009 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:54.954788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb8690-117c-4580-a32a-61bff7e43009 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:54.955449 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb8690-117c-4580-a32a-61bff7e43009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:54.955862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb8690-117c-4580-a32a-61bff7e43009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:54.957402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb8690-117c-4580-a32a-61bff7e43009 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:54.983767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 373/753] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:54.989780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb8690-117c-4580-a32a-61bff7e43009 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:55.039304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:55.051506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb253-2863-466e-9234-0063e50cde2e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:55.089459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fed8c8-65bc-4cd2-8f36-ef8ed6ab8d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:55.091028 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fed8c8-65bc-4cd2-8f36-ef8ed6ab8d95 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:55.091526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fed8c8-65bc-4cd2-8f36-ef8ed6ab8d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:55.091970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fed8c8-65bc-4cd2-8f36-ef8ed6ab8d95 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:55.092827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fed8c8-65bc-4cd2-8f36-ef8ed6ab8d95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:55.093300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fed8c8-65bc-4cd2-8f36-ef8ed6ab8d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:55.094955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fed8c8-65bc-4cd2-8f36-ef8ed6ab8d95 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:55.105003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 381/754] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:55.109909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fed8c8-65bc-4cd2-8f36-ef8ed6ab8d95 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:55.114389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:55.123325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb8690-117c-4580-a32a-61bff7e43009 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:55.160943 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c978a6d-d859-4520-9cad-0277c16a3feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:55.161827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c978a6d-d859-4520-9cad-0277c16a3feb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:55.162289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c978a6d-d859-4520-9cad-0277c16a3feb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:55.162728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c978a6d-d859-4520-9cad-0277c16a3feb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:55.169856 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c978a6d-d859-4520-9cad-0277c16a3feb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:55.171555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c978a6d-d859-4520-9cad-0277c16a3feb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-9c978a6d-d859-4520-9cad-0277c16a3feb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:55.184448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:55.185349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:55.186238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:55.186238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:55.186238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:55.186238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:55.186238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:55.186238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:55.186238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:55.186238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:55.192993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 374/755] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:55.194464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c978a6d-d859-4520-9cad-0277c16a3feb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:55.221334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fed8c8-65bc-4cd2-8f36-ef8ed6ab8d95 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:55.260651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12810b96-d184-46b6-a2d7-9bd0e07e298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:55.261386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12810b96-d184-46b6-a2d7-9bd0e07e298e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:55.262947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12810b96-d184-46b6-a2d7-9bd0e07e298e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:55.262947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12810b96-d184-46b6-a2d7-9bd0e07e298e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:55.274327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12810b96-d184-46b6-a2d7-9bd0e07e298e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:55.275928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12810b96-d184-46b6-a2d7-9bd0e07e298e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:55.292138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-12810b96-d184-46b6-a2d7-9bd0e07e298e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:55.293612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:55.295152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:55.296229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:55.296229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:55.296229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:55.296229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 382/756] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:55.301491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12810b96-d184-46b6-a2d7-9bd0e07e298e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:55.308908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c978a6d-d859-4520-9cad-0277c16a3feb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:55.344172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcc479-8bf1-4228-a35a-5b2c69a90f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:55.344885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcc479-8bf1-4228-a35a-5b2c69a90f9e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:55.345228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcc479-8bf1-4228-a35a-5b2c69a90f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:55.345569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcc479-8bf1-4228-a35a-5b2c69a90f9e None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:55.346353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcc479-8bf1-4228-a35a-5b2c69a90f9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:55.346744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcc479-8bf1-4228-a35a-5b2c69a90f9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:55.348234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcc479-8bf1-4228-a35a-5b2c69a90f9e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:55.365310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 375/757] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:55.365907 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:55.366745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcc479-8bf1-4228-a35a-5b2c69a90f9e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:55.411982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12810b96-d184-46b6-a2d7-9bd0e07e298e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:55.450118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c7e3-d0cd-4742-b331-7ed1a9671510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:55.450773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c7e3-d0cd-4742-b331-7ed1a9671510 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:55.451057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c7e3-d0cd-4742-b331-7ed1a9671510 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:55.451322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c7e3-d0cd-4742-b331-7ed1a9671510 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:55.451951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c7e3-d0cd-4742-b331-7ed1a9671510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:55.452246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c7e3-d0cd-4742-b331-7ed1a9671510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:55.453613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c7e3-d0cd-4742-b331-7ed1a9671510 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:55.467668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 383/758] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:55.473041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c7e3-d0cd-4742-b331-7ed1a9671510 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:55.492564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcc479-8bf1-4228-a35a-5b2c69a90f9e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:55.533199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc65323-25ac-4fdb-b018-4812f5059d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:55.533968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc65323-25ac-4fdb-b018-4812f5059d5b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:55.534264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc65323-25ac-4fdb-b018-4812f5059d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:55.534539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc65323-25ac-4fdb-b018-4812f5059d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:55.535265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc65323-25ac-4fdb-b018-4812f5059d5b None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=ee0fe35d9e304ee495998c93c6909fa8) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:55.535573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc65323-25ac-4fdb-b018-4812f5059d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:55.537176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc65323-25ac-4fdb-b018-4812f5059d5b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:55.550094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 376/759] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:14:55 2022] GET /identity/v3/role_assignments?user.id=ee0fe35d9e304ee495998c93c6909fa8&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:55.675533 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c7e3-d0cd-4742-b331-7ed1a9671510 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:55.698177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:55.724448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673b782-a957-4a3e-8cb5-b7b20f53f68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:55.725109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673b782-a957-4a3e-8cb5-b7b20f53f68f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:55.725384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673b782-a957-4a3e-8cb5-b7b20f53f68f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:55.725653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673b782-a957-4a3e-8cb5-b7b20f53f68f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:55.738516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673b782-a957-4a3e-8cb5-b7b20f53f68f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:55.740005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673b782-a957-4a3e-8cb5-b7b20f53f68f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-6673b782-a957-4a3e-8cb5-b7b20f53f68f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:55.755912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:55.756902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:55.757864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 384/760] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Sep 21 05:14:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:55.763134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc65323-25ac-4fdb-b018-4812f5059d5b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:55.789713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:55.890217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc65323-25ac-4fdb-b018-4812f5059d5b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:55.925791 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72256e8-e58f-4cfb-992d-01149da8c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:55.926350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72256e8-e58f-4cfb-992d-01149da8c8af None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:55.926629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72256e8-e58f-4cfb-992d-01149da8c8af None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:55.926901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72256e8-e58f-4cfb-992d-01149da8c8af None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:55.927626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72256e8-e58f-4cfb-992d-01149da8c8af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:55.927931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72256e8-e58f-4cfb-992d-01149da8c8af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:55.929360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72256e8-e58f-4cfb-992d-01149da8c8af None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:55.948021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 377/761] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Sep 21 05:14:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:55.953211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673b782-a957-4a3e-8cb5-b7b20f53f68f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:56.058960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673b782-a957-4a3e-8cb5-b7b20f53f68f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:56.108740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf34f7-7a15-40f7-9480-ce35df9f910b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:56.109378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf34f7-7a15-40f7-9480-ce35df9f910b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:56.109658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf34f7-7a15-40f7-9480-ce35df9f910b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:56.109947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf34f7-7a15-40f7-9480-ce35df9f910b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:56.127489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf34f7-7a15-40f7-9480-ce35df9f910b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:56.128985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf34f7-7a15-40f7-9480-ce35df9f910b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5adf34f7-7a15-40f7-9480-ce35df9f910b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:56.147233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:56.149527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:56.151575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:56.151575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:56.151575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:56.151575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 385/762] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:56.159188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72256e8-e58f-4cfb-992d-01149da8c8af None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:56.238337 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:56.272721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72256e8-e58f-4cfb-992d-01149da8c8af None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:56.313987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd15b90-f0da-41ab-9d29-7f8998711a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:56.314682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd15b90-f0da-41ab-9d29-7f8998711a1a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:56.314962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd15b90-f0da-41ab-9d29-7f8998711a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:56.315277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd15b90-f0da-41ab-9d29-7f8998711a1a None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:56.315944 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd15b90-f0da-41ab-9d29-7f8998711a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:56.316249 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd15b90-f0da-41ab-9d29-7f8998711a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:56.317721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd15b90-f0da-41ab-9d29-7f8998711a1a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:56.331211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 378/763] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:56.339053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf34f7-7a15-40f7-9480-ce35df9f910b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:56.392990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:56.404205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f3f60-00cb-4766-a154-c4729d2099d6 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:56.404491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f3f60-00cb-4766-a154-c4729d2099d6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:56.404785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f3f60-00cb-4766-a154-c4729d2099d6 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:56.406409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 379/764] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:14:56.417702 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0067487-67cd-4d53-a017-afce7b09eea0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:56.418027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0067487-67cd-4d53-a017-afce7b09eea0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:56.418306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0067487-67cd-4d53-a017-afce7b09eea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:56.450358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b0067487-67cd-4d53-a017-afce7b09eea0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:56.451538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b0067487-67cd-4d53-a017-afce7b09eea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:56.489154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf34f7-7a15-40f7-9480-ce35df9f910b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:56.544193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 380/765] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:14:56.560547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f04695-e495-488e-a6ec-86c75e5452be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:56.561229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f04695-e495-488e-a6ec-86c75e5452be None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:56.561513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f04695-e495-488e-a6ec-86c75e5452be None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:56.561786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f04695-e495-488e-a6ec-86c75e5452be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:56.571923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f04695-e495-488e-a6ec-86c75e5452be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:56.573409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f04695-e495-488e-a6ec-86c75e5452be None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d3f04695-e495-488e-a6ec-86c75e5452be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:14:56.587637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:56.588862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:56.590740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:14:56.590740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:14:56.590740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:14:56.590740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:14:56.590740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:14:56.590740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:14:56.590740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:14:56.590740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:56.590740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 386/766] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:56.594989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0067487-67cd-4d53-a017-afce7b09eea0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:56.701989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:56.763192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0067487-67cd-4d53-a017-afce7b09eea0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:56.803701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de714-ee93-47da-a1db-db4d6ea12eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:56.804296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de714-ee93-47da-a1db-db4d6ea12eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:56.804571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de714-ee93-47da-a1db-db4d6ea12eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:56.804838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de714-ee93-47da-a1db-db4d6ea12eb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:56.805519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de714-ee93-47da-a1db-db4d6ea12eb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:56.805822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de714-ee93-47da-a1db-db4d6ea12eb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:56.807349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de714-ee93-47da-a1db-db4d6ea12eb1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:56.819820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 381/767] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:56.826204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f04695-e495-488e-a6ec-86c75e5452be None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:56.854211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de714-ee93-47da-a1db-db4d6ea12eb1 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:56.988333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:57.001848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f04695-e495-488e-a6ec-86c75e5452be None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:57.039430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9bffc-dc70-4912-b8ad-6510c18b410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:57.040007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9bffc-dc70-4912-b8ad-6510c18b410e None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:57.040289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9bffc-dc70-4912-b8ad-6510c18b410e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:57.040566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9bffc-dc70-4912-b8ad-6510c18b410e None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d9168d8bbf8b43299a7748c9faa099b1/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:57.042490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9bffc-dc70-4912-b8ad-6510c18b410e None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=d9168d8bbf8b43299a7748c9faa099b1, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:57.043964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9bffc-dc70-4912-b8ad-6510c18b410e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:57.050153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de714-ee93-47da-a1db-db4d6ea12eb1 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:57.078040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 387/768] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:14:56 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d9168d8bbf8b43299a7748c9faa099b1/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:14:57.090430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:57.117743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d164975-7044-4cee-bf99-87915fdc3c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:57.118423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d164975-7044-4cee-bf99-87915fdc3c30 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:57.118761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d164975-7044-4cee-bf99-87915fdc3c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:57.119081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d164975-7044-4cee-bf99-87915fdc3c30 None admin] PATH_INFO: `/v3/services/network` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:57.119750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d164975-7044-4cee-bf99-87915fdc3c30 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:57.121610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d164975-7044-4cee-bf99-87915fdc3c30 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5d164975-7044-4cee-bf99-87915fdc3c30 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:57.135583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:57.137757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:57.138928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 382/769] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:56 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:57.142880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9bffc-dc70-4912-b8ad-6510c18b410e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:57.229367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c9c94-36da-44b3-808b-2b9ca2fe34a4 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:57.229640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c9c94-36da-44b3-808b-2b9ca2fe34a4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:57.229913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c9c94-36da-44b3-808b-2b9ca2fe34a4 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:57.232233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 383/770] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:57.237313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de98f08-2c1d-4118-b769-f6c4ebfd8adf None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:57.237851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de98f08-2c1d-4118-b769-f6c4ebfd8adf None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:57.238137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de98f08-2c1d-4118-b769-f6c4ebfd8adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:57.272582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9de98f08-2c1d-4118-b769-f6c4ebfd8adf None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:57.277493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9de98f08-2c1d-4118-b769-f6c4ebfd8adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:57.302294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9bffc-dc70-4912-b8ad-6510c18b410e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:57.369469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 384/771] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:14:57.394029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a3e3f-cb20-494d-bef4-ced174634278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:57.394029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a3e3f-cb20-494d-bef4-ced174634278 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:57.394029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a3e3f-cb20-494d-bef4-ced174634278 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:57.394029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a3e3f-cb20-494d-bef4-ced174634278 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:57.394444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a3e3f-cb20-494d-bef4-ced174634278 None admin] RBAC: Adding query filter params (name=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:57.394444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a3e3f-cb20-494d-bef4-ced174634278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:57.395027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a3e3f-cb20-494d-bef4-ced174634278 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:57.400976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 388/772] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:14:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:57.406327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de98f08-2c1d-4118-b769-f6c4ebfd8adf None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:57.511348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de98f08-2c1d-4118-b769-f6c4ebfd8adf None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:57.525151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:57.546501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cee47-4354-4799-94a8-d0d2cd3dc71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:57.547103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cee47-4354-4799-94a8-d0d2cd3dc71a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:57.547379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cee47-4354-4799-94a8-d0d2cd3dc71a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:57.547651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cee47-4354-4799-94a8-d0d2cd3dc71a None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:57.548270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cee47-4354-4799-94a8-d0d2cd3dc71a None admin] RBAC: Adding query filter params (type=network) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:57.548569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cee47-4354-4799-94a8-d0d2cd3dc71a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:57.549960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cee47-4354-4799-94a8-d0d2cd3dc71a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:57.555210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 385/773] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:14:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:57.560554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a3e3f-cb20-494d-bef4-ced174634278 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:57.592679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:57.688989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cee47-4354-4799-94a8-d0d2cd3dc71a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:57.712910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a3e3f-cb20-494d-bef4-ced174634278 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:57.776980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2801a-990d-4c74-a4fc-9dcc015a6bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:57.778146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2801a-990d-4c74-a4fc-9dcc015a6bba None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:57.778459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2801a-990d-4c74-a4fc-9dcc015a6bba None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:57.778698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2801a-990d-4c74-a4fc-9dcc015a6bba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:57.779393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2801a-990d-4c74-a4fc-9dcc015a6bba None admin] RBAC: Adding query filter params (interface=public, service_id=238dd681d514488ea44be49481313b19, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:57.779700 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2801a-990d-4c74-a4fc-9dcc015a6bba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:57.781051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2801a-990d-4c74-a4fc-9dcc015a6bba None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:57.788927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 389/774] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Sep 21 05:14:57 2022] GET /identity/v3/endpoints?service_id=238dd681d514488ea44be49481313b19&interface=public®ion_id=RegionOne => generated 194 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:57.794933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2801a-990d-4c74-a4fc-9dcc015a6bba None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:57.879705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:57.905371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2801a-990d-4c74-a4fc-9dcc015a6bba None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:57.906785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cee47-4354-4799-94a8-d0d2cd3dc71a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:57.961008 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71d7d-2a55-4a05-a461-e7c42127d681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:57.961625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71d7d-2a55-4a05-a461-e7c42127d681 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:57.961907 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71d7d-2a55-4a05-a461-e7c42127d681 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:57.962201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71d7d-2a55-4a05-a461-e7c42127d681 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:57.962805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb71d7d-2a55-4a05-a461-e7c42127d681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:57.964162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb71d7d-2a55-4a05-a461-e7c42127d681 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:57.974752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 390/775] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Sep 21 05:14:57 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:57.985711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571b7e3-cc40-415d-bf26-eaef3f0bc61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:57.986386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571b7e3-cc40-415d-bf26-eaef3f0bc61f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:57.986667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571b7e3-cc40-415d-bf26-eaef3f0bc61f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:57.986941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571b7e3-cc40-415d-bf26-eaef3f0bc61f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.003851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571b7e3-cc40-415d-bf26-eaef3f0bc61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-d571b7e3-cc40-415d-bf26-eaef3f0bc61f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:58.009472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:58.010479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:58.011286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 386/776] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:14:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:58.017404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71d7d-2a55-4a05-a461-e7c42127d681 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:58.047474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597186f2-740d-45c1-a022-05cfb62d0fa5 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.047796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597186f2-740d-45c1-a022-05cfb62d0fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.048026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597186f2-740d-45c1-a022-05cfb62d0fa5 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.049591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 387/777] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:58.055772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49800c7-6379-4859-8db5-ccdf36003fdb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.056488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49800c7-6379-4859-8db5-ccdf36003fdb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.056488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49800c7-6379-4859-8db5-ccdf36003fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.098471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b49800c7-6379-4859-8db5-ccdf36003fdb None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:58.099853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b49800c7-6379-4859-8db5-ccdf36003fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:58.182426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:58.210187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71d7d-2a55-4a05-a461-e7c42127d681 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:58.235933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 388/778] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:14:58.252892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a30efe-e139-4444-bb4e-a905a5bfb039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:58.253624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a30efe-e139-4444-bb4e-a905a5bfb039 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.253993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a30efe-e139-4444-bb4e-a905a5bfb039 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.254347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a30efe-e139-4444-bb4e-a905a5bfb039 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.255046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a30efe-e139-4444-bb4e-a905a5bfb039 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:58.255354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a30efe-e139-4444-bb4e-a905a5bfb039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:58.256792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a30efe-e139-4444-bb4e-a905a5bfb039 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:58.267533 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 391/779] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:14:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:58.273432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49800c7-6379-4859-8db5-ccdf36003fdb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:58.386157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b995ef-1283-4cb1-838d-303b4ff9d849 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.387027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b995ef-1283-4cb1-838d-303b4ff9d849 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.387470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b995ef-1283-4cb1-838d-303b4ff9d849 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.389282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 392/780] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:58.394623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3f02-a014-47f1-9c6c-ff4c0bfe5023 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.394980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3f02-a014-47f1-9c6c-ff4c0bfe5023 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.395492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3f02-a014-47f1-9c6c-ff4c0bfe5023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.396091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49800c7-6379-4859-8db5-ccdf36003fdb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:58.410509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:58.432009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5c9f3f02-a014-47f1-9c6c-ff4c0bfe5023 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:14:58.433804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5c9f3f02-a014-47f1-9c6c-ff4c0bfe5023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:14:58.451205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45cd70-d4fd-4467-8f5e-d89028cebe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:58.451850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45cd70-d4fd-4467-8f5e-d89028cebe4d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.452129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45cd70-d4fd-4467-8f5e-d89028cebe4d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.452405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45cd70-d4fd-4467-8f5e-d89028cebe4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.465440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45cd70-d4fd-4467-8f5e-d89028cebe4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:58.466927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45cd70-d4fd-4467-8f5e-d89028cebe4d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-4e45cd70-d4fd-4467-8f5e-d89028cebe4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:58.481745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:58.482738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:58.483613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:58.483613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:58.483613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:58.487773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 389/781] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:58.488965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45cd70-d4fd-4467-8f5e-d89028cebe4d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:58.512586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 393/782] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:14:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:14:58.526483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:58.602144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f3f02-a014-47f1-9c6c-ff4c0bfe5023 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:58.644563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45cd70-d4fd-4467-8f5e-d89028cebe4d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:58.685747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12c0f2-17a8-4561-a7c5-e819ca38e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:58.686312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12c0f2-17a8-4561-a7c5-e819ca38e324 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.686589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12c0f2-17a8-4561-a7c5-e819ca38e324 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.687289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12c0f2-17a8-4561-a7c5-e819ca38e324 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.687931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12c0f2-17a8-4561-a7c5-e819ca38e324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:58.688237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12c0f2-17a8-4561-a7c5-e819ca38e324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:58.689971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12c0f2-17a8-4561-a7c5-e819ca38e324 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:58.706030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 390/783] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:14:58.711399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12c0f2-17a8-4561-a7c5-e819ca38e324 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:58.721758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f3f02-a014-47f1-9c6c-ff4c0bfe5023 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:58.774327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:58.780816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6be417-b1cc-4cb8-9800-f2113da3ccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:58.781466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6be417-b1cc-4cb8-9800-f2113da3ccff None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.781747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6be417-b1cc-4cb8-9800-f2113da3ccff None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.782043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6be417-b1cc-4cb8-9800-f2113da3ccff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.782635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6be417-b1cc-4cb8-9800-f2113da3ccff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:58.785541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6be417-b1cc-4cb8-9800-f2113da3ccff None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4e6be417-b1cc-4cb8-9800-f2113da3ccff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:58.804327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:58.805321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:14:58.806180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 394/784] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Sep 21 05:14:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:14:58.811846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6be417-b1cc-4cb8-9800-f2113da3ccff None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:58.830403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:58.852212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12c0f2-17a8-4561-a7c5-e819ca38e324 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:58.891105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d84dcd-8e85-49d0-a59c-38e48c949f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:58.892095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d84dcd-8e85-49d0-a59c-38e48c949f10 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.892609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d84dcd-8e85-49d0-a59c-38e48c949f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.893086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d84dcd-8e85-49d0-a59c-38e48c949f10 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.901129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d84dcd-8e85-49d0-a59c-38e48c949f10 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:58.902939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d84dcd-8e85-49d0-a59c-38e48c949f10 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-d4d84dcd-8e85-49d0-a59c-38e48c949f10 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:14:58.915602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:58.916554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:14:58.917474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:58.920514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 391/785] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:14:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:58.920820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6be417-b1cc-4cb8-9800-f2113da3ccff None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:58.922154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d84dcd-8e85-49d0-a59c-38e48c949f10 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:58.957388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f9e45-e072-41ab-914f-3ca57012eb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:58.958104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f9e45-e072-41ab-914f-3ca57012eb72 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:58.958466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f9e45-e072-41ab-914f-3ca57012eb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:58.958874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f9e45-e072-41ab-914f-3ca57012eb72 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:58.959492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f9e45-e072-41ab-914f-3ca57012eb72 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:58.959888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f9e45-e072-41ab-914f-3ca57012eb72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:58.961503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f9e45-e072-41ab-914f-3ca57012eb72 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:58.971521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 395/786] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Sep 21 05:14:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:58.972495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f9e45-e072-41ab-914f-3ca57012eb72 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.030390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d84dcd-8e85-49d0-a59c-38e48c949f10 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.049443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:59.068013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940eb2b-8fba-40cd-84e1-8a5d5a39d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.069365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940eb2b-8fba-40cd-84e1-8a5d5a39d175 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.069830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940eb2b-8fba-40cd-84e1-8a5d5a39d175 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.070357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940eb2b-8fba-40cd-84e1-8a5d5a39d175 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.081814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940eb2b-8fba-40cd-84e1-8a5d5a39d175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:14:59.090182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c940eb2b-8fba-40cd-84e1-8a5d5a39d175 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:14:59.091511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:59.092719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:14:59.093532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:59.094975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f9e45-e072-41ab-914f-3ca57012eb72 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.100852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 392/787] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:59.101892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940eb2b-8fba-40cd-84e1-8a5d5a39d175 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.131429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9a316-eb32-4596-82b1-3aa0fc27856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.132175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9a316-eb32-4596-82b1-3aa0fc27856c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.132525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9a316-eb32-4596-82b1-3aa0fc27856c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.132912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9a316-eb32-4596-82b1-3aa0fc27856c None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.133708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9a316-eb32-4596-82b1-3aa0fc27856c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:59.134102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9a316-eb32-4596-82b1-3aa0fc27856c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:59.135826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9a316-eb32-4596-82b1-3aa0fc27856c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:59.155892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 396/788] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Sep 21 05:14:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:59.156839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9a316-eb32-4596-82b1-3aa0fc27856c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.209243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940eb2b-8fba-40cd-84e1-8a5d5a39d175 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.247158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df81a8-0d1f-43f8-a362-d51da2f7d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.248125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df81a8-0d1f-43f8-a362-d51da2f7d67d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.248581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df81a8-0d1f-43f8-a362-d51da2f7d67d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.249083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df81a8-0d1f-43f8-a362-d51da2f7d67d None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.250070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df81a8-0d1f-43f8-a362-d51da2f7d67d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:59.250616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df81a8-0d1f-43f8-a362-d51da2f7d67d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:59.252298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df81a8-0d1f-43f8-a362-d51da2f7d67d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:59.258287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 393/789] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:59.259388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df81a8-0d1f-43f8-a362-d51da2f7d67d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.275337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9a316-eb32-4596-82b1-3aa0fc27856c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.295565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:59.314626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956cc7-97bd-489d-9ebe-744547e22cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.315239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956cc7-97bd-489d-9ebe-744547e22cab None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.315630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956cc7-97bd-489d-9ebe-744547e22cab None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.315854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956cc7-97bd-489d-9ebe-744547e22cab None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.316572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43956cc7-97bd-489d-9ebe-744547e22cab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:59.316874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43956cc7-97bd-489d-9ebe-744547e22cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:59.318365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43956cc7-97bd-489d-9ebe-744547e22cab None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:59.332592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 397/790] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:59.332592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956cc7-97bd-489d-9ebe-744547e22cab None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.336833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:59.377876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df81a8-0d1f-43f8-a362-d51da2f7d67d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.414548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22984bda-f2c5-4e0a-a024-b6f72b58bf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.415572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22984bda-f2c5-4e0a-a024-b6f72b58bf03 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.416129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22984bda-f2c5-4e0a-a024-b6f72b58bf03 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.416609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22984bda-f2c5-4e0a-a024-b6f72b58bf03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.429605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22984bda-f2c5-4e0a-a024-b6f72b58bf03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:59.432053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22984bda-f2c5-4e0a-a024-b6f72b58bf03 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:59.447663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956cc7-97bd-489d-9ebe-744547e22cab None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-22984bda-f2c5-4e0a-a024-b6f72b58bf03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:59.448356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:59.449823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:59.450763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:59.450763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:59.450763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:59.452899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 394/791] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:59.453888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22984bda-f2c5-4e0a-a024-b6f72b58bf03 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.486015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cc030-8ad1-41c6-9a68-6c09897d4846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.486602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cc030-8ad1-41c6-9a68-6c09897d4846 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.486882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cc030-8ad1-41c6-9a68-6c09897d4846 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.487156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cc030-8ad1-41c6-9a68-6c09897d4846 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.488167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cc030-8ad1-41c6-9a68-6c09897d4846 None admin] RBAC: Adding query filter params (interface=public, service_id=a2390e6d7d4e45908dff9e7b821b774c, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:59.488481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cc030-8ad1-41c6-9a68-6c09897d4846 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:59.489972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cc030-8ad1-41c6-9a68-6c09897d4846 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:59.500084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 398/792] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/endpoints?service_id=a2390e6d7d4e45908dff9e7b821b774c&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:59.500915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cc030-8ad1-41c6-9a68-6c09897d4846 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.551169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:59.568193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22984bda-f2c5-4e0a-a024-b6f72b58bf03 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.586083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:59.611819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf90e85-d10f-4186-91ca-5255ad1a7e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.613048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cc030-8ad1-41c6-9a68-6c09897d4846 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.613251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf90e85-d10f-4186-91ca-5255ad1a7e58 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.613717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf90e85-d10f-4186-91ca-5255ad1a7e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.614236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf90e85-d10f-4186-91ca-5255ad1a7e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.625889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf90e85-d10f-4186-91ca-5255ad1a7e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:59.627752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf90e85-d10f-4186-91ca-5255ad1a7e58 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5bf90e85-d10f-4186-91ca-5255ad1a7e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:14:59.645330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:14:59.646325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:14:59.647187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:14:59.647187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:14:59.647187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:14:59.648950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 395/793] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:14:59.649461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f614-612c-4ad1-9a4c-d7d3bd1fdfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.650116 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f614-612c-4ad1-9a4c-d7d3bd1fdfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.650427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf90e85-d10f-4186-91ca-5255ad1a7e58 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.650541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f614-612c-4ad1-9a4c-d7d3bd1fdfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.650776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f614-612c-4ad1-9a4c-d7d3bd1fdfb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.651642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4f614-612c-4ad1-9a4c-d7d3bd1fdfb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:59.651923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4f614-612c-4ad1-9a4c-d7d3bd1fdfb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:59.653919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4f614-612c-4ad1-9a4c-d7d3bd1fdfb0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:59.667824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 399/794] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:59.668857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f614-612c-4ad1-9a4c-d7d3bd1fdfb0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.768696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf90e85-d10f-4186-91ca-5255ad1a7e58 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.781799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f614-612c-4ad1-9a4c-d7d3bd1fdfb0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:14:59.784478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:59.806301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95296d2d-9cc9-4ddb-8701-e79dd1bd399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.807170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95296d2d-9cc9-4ddb-8701-e79dd1bd399f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.807609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95296d2d-9cc9-4ddb-8701-e79dd1bd399f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.807867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95296d2d-9cc9-4ddb-8701-e79dd1bd399f None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.809020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95296d2d-9cc9-4ddb-8701-e79dd1bd399f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:59.810396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95296d2d-9cc9-4ddb-8701-e79dd1bd399f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:59.815848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 396/795] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:59.816469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95296d2d-9cc9-4ddb-8701-e79dd1bd399f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.831272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663675e-6f9b-45e7-a076-7341bbc28870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:14:59.831903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663675e-6f9b-45e7-a076-7341bbc28870 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.832213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663675e-6f9b-45e7-a076-7341bbc28870 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.832491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663675e-6f9b-45e7-a076-7341bbc28870 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.833139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663675e-6f9b-45e7-a076-7341bbc28870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:14:59.833445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663675e-6f9b-45e7-a076-7341bbc28870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:14:59.834834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663675e-6f9b-45e7-a076-7341bbc28870 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:14:59.855844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 400/796] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:14:59.855844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f916ba-e6fa-4d58-a647-7ae504142d8b None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:14:59.855844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f916ba-e6fa-4d58-a647-7ae504142d8b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:14:59.856186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f916ba-e6fa-4d58-a647-7ae504142d8b None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:14:59.857561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 401/797] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:14:59.861811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f916ba-e6fa-4d58-a647-7ae504142d8b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:14:59.862923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:14:59.961402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95296d2d-9cc9-4ddb-8701-e79dd1bd399f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:00.000030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac08e8f-d8d5-4e8e-ac76-519e1b163891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:00.000662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac08e8f-d8d5-4e8e-ac76-519e1b163891 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.000937 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac08e8f-d8d5-4e8e-ac76-519e1b163891 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.001204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac08e8f-d8d5-4e8e-ac76-519e1b163891 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.005252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f916ba-e6fa-4d58-a647-7ae504142d8b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:00.008416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac08e8f-d8d5-4e8e-ac76-519e1b163891 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:00.010217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac08e8f-d8d5-4e8e-ac76-519e1b163891 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-6ac08e8f-d8d5-4e8e-ac76-519e1b163891 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:00.023736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:00.024621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:00.025530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:00.025530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:00.025530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:00.025530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:00.025530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:00.025530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:00.025530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:00.025530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:00.027745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 397/798] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:00.033064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb29ab-c232-481c-b80c-d2accbda72ba None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.033064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb29ab-c232-481c-b80c-d2accbda72ba None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.033064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb29ab-c232-481c-b80c-d2accbda72ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.047286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d2f1a-3aad-4760-b3e4-19533e7fc9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:00.047921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d2f1a-3aad-4760-b3e4-19533e7fc9fa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.048221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d2f1a-3aad-4760-b3e4-19533e7fc9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.048472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d2f1a-3aad-4760-b3e4-19533e7fc9fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.053640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5dfb29ab-c232-481c-b80c-d2accbda72ba None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:00.055078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5dfb29ab-c232-481c-b80c-d2accbda72ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:00.057159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d2f1a-3aad-4760-b3e4-19533e7fc9fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:00.059787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d2f1a-3aad-4760-b3e4-19533e7fc9fa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c31d2f1a-3aad-4760-b3e4-19533e7fc9fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:00.071165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:00.072318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 402/799] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:14:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:00.073478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d2f1a-3aad-4760-b3e4-19533e7fc9fa None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:00.111408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:00.142257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 398/800] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:00.143295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb29ab-c232-481c-b80c-d2accbda72ba None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:00.202420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:00.274782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb29ab-c232-481c-b80c-d2accbda72ba None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:00.311475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d2f1a-3aad-4760-b3e4-19533e7fc9fa None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:00.312687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c4406-ea30-4fa0-9502-eb04004cc8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:00.313672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c4406-ea30-4fa0-9502-eb04004cc8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.314184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c4406-ea30-4fa0-9502-eb04004cc8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.314655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c4406-ea30-4fa0-9502-eb04004cc8d4 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.315586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c4406-ea30-4fa0-9502-eb04004cc8d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:00.316102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c4406-ea30-4fa0-9502-eb04004cc8d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:00.318591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c4406-ea30-4fa0-9502-eb04004cc8d4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:00.344972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 399/801] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Sep 21 05:15:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:00.350675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c4406-ea30-4fa0-9502-eb04004cc8d4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:00.385765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c23897-daa0-45c4-82a3-49b8f805fdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:00.387870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c23897-daa0-45c4-82a3-49b8f805fdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.389806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c23897-daa0-45c4-82a3-49b8f805fdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.390324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c23897-daa0-45c4-82a3-49b8f805fdc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.391223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c23897-daa0-45c4-82a3-49b8f805fdc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:00.391733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c23897-daa0-45c4-82a3-49b8f805fdc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:00.395183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c23897-daa0-45c4-82a3-49b8f805fdc0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:00.430443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 403/802] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:00.430443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c23897-daa0-45c4-82a3-49b8f805fdc0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:00.486019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:00.506440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c4406-ea30-4fa0-9502-eb04004cc8d4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:00.530274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:00.542518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ebcc1-9d92-49d4-b767-2095a459fcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:00.544049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ebcc1-9d92-49d4-b767-2095a459fcd9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.544532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ebcc1-9d92-49d4-b767-2095a459fcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.545002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ebcc1-9d92-49d4-b767-2095a459fcd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.549587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c23897-daa0-45c4-82a3-49b8f805fdc0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:00.556300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ebcc1-9d92-49d4-b767-2095a459fcd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:00.558002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ebcc1-9d92-49d4-b767-2095a459fcd9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-4e2ebcc1-9d92-49d4-b767-2095a459fcd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:00.573538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:00.574606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:00.575470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:00.575470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:00.575470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:00.575470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 400/803] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:00.575470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ebcc1-9d92-49d4-b767-2095a459fcd9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:00.595341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747da7e-f0d4-4db6-9034-cbd270da75fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:00.595980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747da7e-f0d4-4db6-9034-cbd270da75fc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.596264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747da7e-f0d4-4db6-9034-cbd270da75fc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.596536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747da7e-f0d4-4db6-9034-cbd270da75fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.607610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747da7e-f0d4-4db6-9034-cbd270da75fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f747da7e-f0d4-4db6-9034-cbd270da75fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:00.612436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:00.613653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:00.614519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 404/804] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:15:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:00.615208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba64577-1847-43f5-aedd-54c00b110905 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.615275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba64577-1847-43f5-aedd-54c00b110905 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.615699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba64577-1847-43f5-aedd-54c00b110905 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.617854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 405/805] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:00.618594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba64577-1847-43f5-aedd-54c00b110905 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:00.692724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ebcc1-9d92-49d4-b767-2095a459fcd9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:00.726343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:00.732893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e570882-6f8f-4fb5-abd4-6433b07f9f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:00.733551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e570882-6f8f-4fb5-abd4-6433b07f9f95 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.733834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e570882-6f8f-4fb5-abd4-6433b07f9f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.734167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e570882-6f8f-4fb5-abd4-6433b07f9f95 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d2297673f49b4ce8bf40ac5f750d989f/roles/99269efe3b2a430e9cc7912268d2be84` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.738228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba64577-1847-43f5-aedd-54c00b110905 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:00.750681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e570882-6f8f-4fb5-abd4-6433b07f9f95 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=d2297673f49b4ce8bf40ac5f750d989f, role_id=99269efe3b2a430e9cc7912268d2be84)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:00.752133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e570882-6f8f-4fb5-abd4-6433b07f9f95 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:00.767159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 401/806] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:15:00 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d2297673f49b4ce8bf40ac5f750d989f/roles/99269efe3b2a430e9cc7912268d2be84 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:15:00.768436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8444af0-9e4e-4b57-bd81-13edbbec40e2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.768724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8444af0-9e4e-4b57-bd81-13edbbec40e2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.768999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8444af0-9e4e-4b57-bd81-13edbbec40e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.789270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e8444af0-9e4e-4b57-bd81-13edbbec40e2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:00.790522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e8444af0-9e4e-4b57-bd81-13edbbec40e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:00.798133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0973d3-bfe9-42fa-b564-a35d74b91528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:00.799177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0973d3-bfe9-42fa-b564-a35d74b91528 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:00.799668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0973d3-bfe9-42fa-b564-a35d74b91528 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:00.800940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0973d3-bfe9-42fa-b564-a35d74b91528 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:00.801919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0973d3-bfe9-42fa-b564-a35d74b91528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:00.802530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0973d3-bfe9-42fa-b564-a35d74b91528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:00.805285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0973d3-bfe9-42fa-b564-a35d74b91528 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:00.823666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 406/807] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:00.825537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0973d3-bfe9-42fa-b564-a35d74b91528 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:00.830401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:00.885480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 402/808] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:00.887458 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8444af0-9e4e-4b57-bd81-13edbbec40e2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:01.021709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8444af0-9e4e-4b57-bd81-13edbbec40e2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:01.037634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:01.062372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96784737-63fd-4677-905e-b2630ed7be34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:01.063387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96784737-63fd-4677-905e-b2630ed7be34 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:01.063874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96784737-63fd-4677-905e-b2630ed7be34 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:01.064371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96784737-63fd-4677-905e-b2630ed7be34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:01.068635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0973d3-bfe9-42fa-b564-a35d74b91528 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:01.072496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96784737-63fd-4677-905e-b2630ed7be34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:01.074799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96784737-63fd-4677-905e-b2630ed7be34 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-96784737-63fd-4677-905e-b2630ed7be34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:01.089144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:01.090776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:01.091653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:01.091653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:01.091653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:01.091653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:01.091653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:01.091653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:01.091653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:01.091653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:01.094908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 403/809] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:01.100852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96784737-63fd-4677-905e-b2630ed7be34 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:01.112433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46f7ba-c6de-4633-b988-208579bbc374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:01.113485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46f7ba-c6de-4633-b988-208579bbc374 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:01.117975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46f7ba-c6de-4633-b988-208579bbc374 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:01.117975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46f7ba-c6de-4633-b988-208579bbc374 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:01.117975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46f7ba-c6de-4633-b988-208579bbc374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:01.117975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46f7ba-c6de-4633-b988-208579bbc374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:01.117975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46f7ba-c6de-4633-b988-208579bbc374 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:01.129755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 407/810] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:15:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:01.137434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46f7ba-c6de-4633-b988-208579bbc374 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:01.260964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:01.278413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96784737-63fd-4677-905e-b2630ed7be34 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:01.306338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46f7ba-c6de-4633-b988-208579bbc374 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:01.326676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797278b-8f54-456d-8e22-1cc1eb927594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:01.327335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797278b-8f54-456d-8e22-1cc1eb927594 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:01.327687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797278b-8f54-456d-8e22-1cc1eb927594 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:01.328033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797278b-8f54-456d-8e22-1cc1eb927594 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:01.328928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797278b-8f54-456d-8e22-1cc1eb927594 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:01.329304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797278b-8f54-456d-8e22-1cc1eb927594 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:01.330912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797278b-8f54-456d-8e22-1cc1eb927594 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:01.344411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 404/811] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:01.345538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797278b-8f54-456d-8e22-1cc1eb927594 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:01.347642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8f8a1-548f-42af-ba49-3827571f5b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:01.348474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8f8a1-548f-42af-ba49-3827571f5b50 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:01.348953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8f8a1-548f-42af-ba49-3827571f5b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:01.349459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8f8a1-548f-42af-ba49-3827571f5b50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:01.360191 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8f8a1-548f-42af-ba49-3827571f5b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:01.361896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8f8a1-548f-42af-ba49-3827571f5b50 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-03e8f8a1-548f-42af-ba49-3827571f5b50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:01.377629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:01.378761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:01.379643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:01.379643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:01.379643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:01.379837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:01.382196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 408/812] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:01.386195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8f8a1-548f-42af-ba49-3827571f5b50 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:01.477526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797278b-8f54-456d-8e22-1cc1eb927594 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:01.500766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8f8a1-548f-42af-ba49-3827571f5b50 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:01.516270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0379ea2-9986-4add-b90a-33845ee613d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:01.517029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0379ea2-9986-4add-b90a-33845ee613d8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:01.517447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0379ea2-9986-4add-b90a-33845ee613d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:01.517819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0379ea2-9986-4add-b90a-33845ee613d8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:01.518562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0379ea2-9986-4add-b90a-33845ee613d8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:01.520162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0379ea2-9986-4add-b90a-33845ee613d8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:01.524911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c0379ea2-9986-4add-b90a-33845ee613d8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:01.529754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:01.531129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 05:15:01.532191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:01.538817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 405/813] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:01 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:01.538817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0379ea2-9986-4add-b90a-33845ee613d8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:01.554545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69649b6-19ef-42a4-9b17-c8780461a224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:01.555706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69649b6-19ef-42a4-9b17-c8780461a224 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:01.556135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69649b6-19ef-42a4-9b17-c8780461a224 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:01.556796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69649b6-19ef-42a4-9b17-c8780461a224 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/ee0fe35d9e304ee495998c93c6909fa8/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:01.559900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69649b6-19ef-42a4-9b17-c8780461a224 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=ee0fe35d9e304ee495998c93c6909fa8, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:01.562103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69649b6-19ef-42a4-9b17-c8780461a224 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:01.580207 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 409/814] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:15:01 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/ee0fe35d9e304ee495998c93c6909fa8/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:15:01.581230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69649b6-19ef-42a4-9b17-c8780461a224 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:01.674076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:01.684774 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0379ea2-9986-4add-b90a-33845ee613d8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:01.736632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158134b2-69e6-4ddb-b1b5-2a1705e520ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:01.737842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158134b2-69e6-4ddb-b1b5-2a1705e520ae None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:01.738287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158134b2-69e6-4ddb-b1b5-2a1705e520ae None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:01.738705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158134b2-69e6-4ddb-b1b5-2a1705e520ae None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:01.739578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158134b2-69e6-4ddb-b1b5-2a1705e520ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:01.740047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158134b2-69e6-4ddb-b1b5-2a1705e520ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:01.741761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158134b2-69e6-4ddb-b1b5-2a1705e520ae None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:01.772121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69649b6-19ef-42a4-9b17-c8780461a224 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:01.780244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 406/815] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:01.791041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158134b2-69e6-4ddb-b1b5-2a1705e520ae None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:01.823555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6803db0-e122-413b-a277-21404b7d9e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:01.825672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6803db0-e122-413b-a277-21404b7d9e69 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:01.826775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6803db0-e122-413b-a277-21404b7d9e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:01.828339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6803db0-e122-413b-a277-21404b7d9e69 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:01.829167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6803db0-e122-413b-a277-21404b7d9e69 None admin] RBAC: Adding query filter params (name=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:01.829830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6803db0-e122-413b-a277-21404b7d9e69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:01.831794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6803db0-e122-413b-a277-21404b7d9e69 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:01.840587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 410/816] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:15:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:01.846190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6803db0-e122-413b-a277-21404b7d9e69 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:01.903380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:01.979886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158134b2-69e6-4ddb-b1b5-2a1705e520ae None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:02.015522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725c49-34ed-4a59-9c24-45ba59077e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:02.016160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725c49-34ed-4a59-9c24-45ba59077e14 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:02.016439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725c49-34ed-4a59-9c24-45ba59077e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:02.016713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725c49-34ed-4a59-9c24-45ba59077e14 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:02.024108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725c49-34ed-4a59-9c24-45ba59077e14 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:02.025549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725c49-34ed-4a59-9c24-45ba59077e14 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:02.035122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6803db0-e122-413b-a277-21404b7d9e69 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-ba725c49-34ed-4a59-9c24-45ba59077e14 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:02.038462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:02.039379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:02.040301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 407/817] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Sep 21 05:15:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:02.046117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725c49-34ed-4a59-9c24-45ba59077e14 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:02.065558 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:02.104909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ff75a-c6c7-4f3a-921e-2627f9b0734e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:02.106147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ff75a-c6c7-4f3a-921e-2627f9b0734e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:02.106429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ff75a-c6c7-4f3a-921e-2627f9b0734e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:02.106700 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ff75a-c6c7-4f3a-921e-2627f9b0734e None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:02.109968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ff75a-c6c7-4f3a-921e-2627f9b0734e None admin] RBAC: Adding query filter params (type=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:02.109968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ff75a-c6c7-4f3a-921e-2627f9b0734e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:02.109968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ff75a-c6c7-4f3a-921e-2627f9b0734e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:02.119834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 411/818] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:15:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:02.127883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ff75a-c6c7-4f3a-921e-2627f9b0734e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:02.213373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725c49-34ed-4a59-9c24-45ba59077e14 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:02.256521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863677ab-2c2f-4ebb-8bb8-b7be8ee9e762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:02.257376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863677ab-2c2f-4ebb-8bb8-b7be8ee9e762 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:02.257724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863677ab-2c2f-4ebb-8bb8-b7be8ee9e762 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:02.258421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863677ab-2c2f-4ebb-8bb8-b7be8ee9e762 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:02.259457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863677ab-2c2f-4ebb-8bb8-b7be8ee9e762 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:02.259849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863677ab-2c2f-4ebb-8bb8-b7be8ee9e762 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:02.261904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863677ab-2c2f-4ebb-8bb8-b7be8ee9e762 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:02.262436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ff75a-c6c7-4f3a-921e-2627f9b0734e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:02.274936 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:02.291030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 408/819] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Sep 21 05:15:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:02.292730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbbf6-68ec-4efe-8a3c-db6b9216a030 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:02.293090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbbf6-68ec-4efe-8a3c-db6b9216a030 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:02.293432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbbf6-68ec-4efe-8a3c-db6b9216a030 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:02.295358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 409/820] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:02.296857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cbbf6-68ec-4efe-8a3c-db6b9216a030 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:02.315976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6f2d6-df50-41e2-9ec2-766e79281b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:02.316780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6f2d6-df50-41e2-9ec2-766e79281b26 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:02.317068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6f2d6-df50-41e2-9ec2-766e79281b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:02.317340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6f2d6-df50-41e2-9ec2-766e79281b26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:02.318033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6f2d6-df50-41e2-9ec2-766e79281b26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:02.320105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6f2d6-df50-41e2-9ec2-766e79281b26 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:02.348796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 412/821] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:15:02 2022] POST /identity/v3/endpoints => generated 337 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:15:02.350309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56313-d5e5-44c4-82b5-1b0125a4def7 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:02.350600 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56313-d5e5-44c4-82b5-1b0125a4def7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:02.350874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56313-d5e5-44c4-82b5-1b0125a4def7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:02.396423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5bb56313-d5e5-44c4-82b5-1b0125a4def7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:02.400018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5bb56313-d5e5-44c4-82b5-1b0125a4def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:02.443310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:02.508773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 413/822] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:02.520229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cbbf6-68ec-4efe-8a3c-db6b9216a030 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:02.563261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd640c-cd2d-481c-96d9-931fc680181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:02.563870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd640c-cd2d-481c-96d9-931fc680181f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:02.564152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd640c-cd2d-481c-96d9-931fc680181f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:02.564423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd640c-cd2d-481c-96d9-931fc680181f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:02.638619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd640c-cd2d-481c-96d9-931fc680181f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:02.640141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd640c-cd2d-481c-96d9-931fc680181f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:02.721347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:02.762654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 410/823] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:02 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:15:02.768339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56313-d5e5-44c4-82b5-1b0125a4def7 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:02.800275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd640c-cd2d-481c-96d9-931fc680181f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:02.884548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:02.901281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56313-d5e5-44c4-82b5-1b0125a4def7 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:02.946659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9db79-e375-4d67-949a-e143398d5020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:02.947305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9db79-e375-4d67-949a-e143398d5020 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:02.947585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9db79-e375-4d67-949a-e143398d5020 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:02.947857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9db79-e375-4d67-949a-e143398d5020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:02.959505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9db79-e375-4d67-949a-e143398d5020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:02.961028 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9db79-e375-4d67-949a-e143398d5020 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-65d9db79-e375-4d67-949a-e143398d5020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:02.978512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:02.979475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:02.980925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:02.980925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:02.980925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:02.983126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 414/824] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:02.989148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9db79-e375-4d67-949a-e143398d5020 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:03.076723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd640c-cd2d-481c-96d9-931fc680181f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:03.107996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9db79-e375-4d67-949a-e143398d5020 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:03.117348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8c5a-dc27-4982-90bf-36d8605a5bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:03.118004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8c5a-dc27-4982-90bf-36d8605a5bff None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.118285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8c5a-dc27-4982-90bf-36d8605a5bff None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.118577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8c5a-dc27-4982-90bf-36d8605a5bff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.119249 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d8c5a-dc27-4982-90bf-36d8605a5bff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:03.120745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d8c5a-dc27-4982-90bf-36d8605a5bff None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c46d8c5a-dc27-4982-90bf-36d8605a5bff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:03.130442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:03.131409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:03.132218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 411/825] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Sep 21 05:15:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:03.139957 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8c5a-dc27-4982-90bf-36d8605a5bff None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:03.156964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54721268-4fbd-423e-8eb5-bd0511122fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:03.157629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54721268-4fbd-423e-8eb5-bd0511122fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.157915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54721268-4fbd-423e-8eb5-bd0511122fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.158213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54721268-4fbd-423e-8eb5-bd0511122fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.159168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54721268-4fbd-423e-8eb5-bd0511122fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:03.159474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54721268-4fbd-423e-8eb5-bd0511122fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:03.161593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54721268-4fbd-423e-8eb5-bd0511122fb0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:03.175016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:03.184949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 415/826] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:03.185102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:03.186890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54721268-4fbd-423e-8eb5-bd0511122fb0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:03.263662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8c5a-dc27-4982-90bf-36d8605a5bff None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:03.294524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54721268-4fbd-423e-8eb5-bd0511122fb0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:03.302098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb431eb-ec1a-4df7-8c1e-b5289366f4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:03.302706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb431eb-ec1a-4df7-8c1e-b5289366f4dc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.302984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb431eb-ec1a-4df7-8c1e-b5289366f4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.303254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb431eb-ec1a-4df7-8c1e-b5289366f4dc None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.303992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb431eb-ec1a-4df7-8c1e-b5289366f4dc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:03.304338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb431eb-ec1a-4df7-8c1e-b5289366f4dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:03.305754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb431eb-ec1a-4df7-8c1e-b5289366f4dc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:03.315668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 412/827] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Sep 21 05:15:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:03.316302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb431eb-ec1a-4df7-8c1e-b5289366f4dc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:03.343109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4654aa-453f-4213-a6e2-622991f2c3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:03.344274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4654aa-453f-4213-a6e2-622991f2c3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.344610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4654aa-453f-4213-a6e2-622991f2c3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.344884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4654aa-453f-4213-a6e2-622991f2c3f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.354450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4654aa-453f-4213-a6e2-622991f2c3f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:03.356651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4654aa-453f-4213-a6e2-622991f2c3f2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3c4654aa-453f-4213-a6e2-622991f2c3f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:03.370788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:03.371694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:03.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:03.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:03.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:03.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:03.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:03.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:03.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:03.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:03.372563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 416/828] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:03.373118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8246e82-c28d-454c-8c63-fe4a8f4eaf9f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.373663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8246e82-c28d-454c-8c63-fe4a8f4eaf9f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.373663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8246e82-c28d-454c-8c63-fe4a8f4eaf9f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.375166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 417/829] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:03.377763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8246e82-c28d-454c-8c63-fe4a8f4eaf9f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:03.430321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:03.434742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb431eb-ec1a-4df7-8c1e-b5289366f4dc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:03.440484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:03.473146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bb03d-6eb9-4229-acd4-9f90202c1c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:03.473746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bb03d-6eb9-4229-acd4-9f90202c1c0a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.474488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bb03d-6eb9-4229-acd4-9f90202c1c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.474766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bb03d-6eb9-4229-acd4-9f90202c1c0a None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.475392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bb03d-6eb9-4229-acd4-9f90202c1c0a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:03.475691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bb03d-6eb9-4229-acd4-9f90202c1c0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:03.477036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bb03d-6eb9-4229-acd4-9f90202c1c0a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:03.486303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 413/830] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Sep 21 05:15:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:03.486303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cee44-6062-40bd-a371-a6b292786333 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.486303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cee44-6062-40bd-a371-a6b292786333 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.486303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cee44-6062-40bd-a371-a6b292786333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.492025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8246e82-c28d-454c-8c63-fe4a8f4eaf9f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:03.501835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f34cee44-6062-40bd-a371-a6b292786333 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:03.503029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f34cee44-6062-40bd-a371-a6b292786333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:03.528263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d7039-d5a5-423e-b512-42a8feda034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:03.528796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d7039-d5a5-423e-b512-42a8feda034b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.529073 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d7039-d5a5-423e-b512-42a8feda034b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.529343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d7039-d5a5-423e-b512-42a8feda034b None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.530049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d7039-d5a5-423e-b512-42a8feda034b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:03.530350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d7039-d5a5-423e-b512-42a8feda034b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:03.531747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d7039-d5a5-423e-b512-42a8feda034b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:03.544162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 418/831] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:03.544958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d7039-d5a5-423e-b512-42a8feda034b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:03.576240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 414/832] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:03.577380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cee44-6062-40bd-a371-a6b292786333 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:03.661507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d7039-d5a5-423e-b512-42a8feda034b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:03.684567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:03.685944 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:03.693966 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cee44-6062-40bd-a371-a6b292786333 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:03.702460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45294a2c-ca6e-472f-b63a-22f64f8dca05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:03.704130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45294a2c-ca6e-472f-b63a-22f64f8dca05 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.704634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45294a2c-ca6e-472f-b63a-22f64f8dca05 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.705186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45294a2c-ca6e-472f-b63a-22f64f8dca05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.706064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45294a2c-ca6e-472f-b63a-22f64f8dca05 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:03.707732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45294a2c-ca6e-472f-b63a-22f64f8dca05 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:03.731749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 419/833] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:15:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:03.740961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b57f56-9308-4822-b044-757956dcc4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:03.740961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b57f56-9308-4822-b044-757956dcc4fc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:03.740961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b57f56-9308-4822-b044-757956dcc4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:03.740961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b57f56-9308-4822-b044-757956dcc4fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:03.741388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b57f56-9308-4822-b044-757956dcc4fc None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d9168d8bbf8b43299a7748c9faa099b1) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:03.741388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b57f56-9308-4822-b044-757956dcc4fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:03.742957 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b57f56-9308-4822-b044-757956dcc4fc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:03.758200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 415/834] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:03 2022] GET /identity/v3/role_assignments?user.id=d9168d8bbf8b43299a7748c9faa099b1&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:03.859244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45294a2c-ca6e-472f-b63a-22f64f8dca05 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:03.979443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45294a2c-ca6e-472f-b63a-22f64f8dca05 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:04.022126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfd700-6e36-4baa-abc4-0ca6b71af715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:04.022766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfd700-6e36-4baa-abc4-0ca6b71af715 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.023047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfd700-6e36-4baa-abc4-0ca6b71af715 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.023317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfd700-6e36-4baa-abc4-0ca6b71af715 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.034525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfd700-6e36-4baa-abc4-0ca6b71af715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-8acfd700-6e36-4baa-abc4-0ca6b71af715 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:04.039735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:04.040625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:04.041482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 420/835] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:15:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:04.047289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b57f56-9308-4822-b044-757956dcc4fc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:04.094254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a05c2-4193-422e-a5b0-72ee746673a0 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.094749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a05c2-4193-422e-a5b0-72ee746673a0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.095161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a05c2-4193-422e-a5b0-72ee746673a0 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.096828 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 421/836] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:04.102389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34055c7-7f2d-45d5-861d-fc30a9751665 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.102678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34055c7-7f2d-45d5-861d-fc30a9751665 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.102956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34055c7-7f2d-45d5-861d-fc30a9751665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.124260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a34055c7-7f2d-45d5-861d-fc30a9751665 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:04.125373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a34055c7-7f2d-45d5-861d-fc30a9751665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:04.157430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:04.158349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b57f56-9308-4822-b044-757956dcc4fc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:04.206096 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 422/837] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:04.206483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96957be-e710-415c-95f6-d516af1aa306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:04.207079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96957be-e710-415c-95f6-d516af1aa306 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.207354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96957be-e710-415c-95f6-d516af1aa306 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.207622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96957be-e710-415c-95f6-d516af1aa306 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.208285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96957be-e710-415c-95f6-d516af1aa306 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:04.208583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96957be-e710-415c-95f6-d516af1aa306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:04.209950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96957be-e710-415c-95f6-d516af1aa306 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:04.219243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 416/838] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:15:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:04.224546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34055c7-7f2d-45d5-861d-fc30a9751665 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:04.329582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34055c7-7f2d-45d5-861d-fc30a9751665 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:04.365054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2395c0c-f199-4e79-9b48-203742e75e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:04.365681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395c0c-f199-4e79-9b48-203742e75e82 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.365976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395c0c-f199-4e79-9b48-203742e75e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.366252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395c0c-f199-4e79-9b48-203742e75e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.376787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2395c0c-f199-4e79-9b48-203742e75e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:04.378163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2395c0c-f199-4e79-9b48-203742e75e82 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a2395c0c-f199-4e79-9b48-203742e75e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:04.394385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:04.396309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:04.398184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:04.398184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:04.398184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:04.398184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 423/839] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:04.401389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96957be-e710-415c-95f6-d516af1aa306 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:04.418676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:04.479836 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2395c0c-f199-4e79-9b48-203742e75e82 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:04.501352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:04.514369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96957be-e710-415c-95f6-d516af1aa306 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:04.550825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bc238-d9bb-4860-add5-9af057ca0682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:04.551806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bc238-d9bb-4860-add5-9af057ca0682 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.552286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bc238-d9bb-4860-add5-9af057ca0682 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.552761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bc238-d9bb-4860-add5-9af057ca0682 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.553628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bc238-d9bb-4860-add5-9af057ca0682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:04.554162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bc238-d9bb-4860-add5-9af057ca0682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:04.555735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bc238-d9bb-4860-add5-9af057ca0682 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:04.569310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 417/840] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:04.574867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bc238-d9bb-4860-add5-9af057ca0682 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:04.629894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2395c0c-f199-4e79-9b48-203742e75e82 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:04.662677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:04.678241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f77df-aee6-4882-9cd6-4ec7918c8756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:04.678974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f77df-aee6-4882-9cd6-4ec7918c8756 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.679325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f77df-aee6-4882-9cd6-4ec7918c8756 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.679674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f77df-aee6-4882-9cd6-4ec7918c8756 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.692542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f77df-aee6-4882-9cd6-4ec7918c8756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:04.693262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bc238-d9bb-4860-add5-9af057ca0682 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-fc9f77df-aee6-4882-9cd6-4ec7918c8756 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:04.697820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:04.699473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:04.700341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 424/841] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:15:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:04.705655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f77df-aee6-4882-9cd6-4ec7918c8756 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:04.744355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8964f-7870-4bb9-bb1e-f89719406c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:04.745910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8964f-7870-4bb9-bb1e-f89719406c92 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.746564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8964f-7870-4bb9-bb1e-f89719406c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.747111 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8964f-7870-4bb9-bb1e-f89719406c92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.757814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8964f-7870-4bb9-bb1e-f89719406c92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:04.760334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8964f-7870-4bb9-bb1e-f89719406c92 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5db8964f-7870-4bb9-bb1e-f89719406c92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:04.780098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:04.781058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:15:04.781970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:04.789571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 418/842] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:15:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:04.789571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8964f-7870-4bb9-bb1e-f89719406c92 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:04.813345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:04.840282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f77df-aee6-4882-9cd6-4ec7918c8756 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:04.881591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fd087-1d8d-4a0a-8226-a45a85b32e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:04.882177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fd087-1d8d-4a0a-8226-a45a85b32e84 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.882461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fd087-1d8d-4a0a-8226-a45a85b32e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.882885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fd087-1d8d-4a0a-8226-a45a85b32e84 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.883558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fd087-1d8d-4a0a-8226-a45a85b32e84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:04.883862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fd087-1d8d-4a0a-8226-a45a85b32e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:04.886046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fd087-1d8d-4a0a-8226-a45a85b32e84 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:04.899831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 425/843] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:15:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:04.900688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fd087-1d8d-4a0a-8226-a45a85b32e84 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:04.930726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8964f-7870-4bb9-bb1e-f89719406c92 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:04.975639 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb57ea-295e-4105-8f65-5075793b2ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:04.977063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb57ea-295e-4105-8f65-5075793b2ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:04.977749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb57ea-295e-4105-8f65-5075793b2ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:04.978293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb57ea-295e-4105-8f65-5075793b2ca6 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:04.979401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb57ea-295e-4105-8f65-5075793b2ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:04.979853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb57ea-295e-4105-8f65-5075793b2ca6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:04.982347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb57ea-295e-4105-8f65-5075793b2ca6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:04.984126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:05.012975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 419/844] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Sep 21 05:15:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:05.013618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb57ea-295e-4105-8f65-5075793b2ca6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:05.023173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fd087-1d8d-4a0a-8226-a45a85b32e84 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:05.065525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3434428-382c-4621-9718-d0a8aa2fd7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:05.066186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3434428-382c-4621-9718-d0a8aa2fd7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:05.066468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3434428-382c-4621-9718-d0a8aa2fd7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:05.066794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3434428-382c-4621-9718-d0a8aa2fd7d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:05.078064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:05.086338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3434428-382c-4621-9718-d0a8aa2fd7d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:05.087775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3434428-382c-4621-9718-d0a8aa2fd7d5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f3434428-382c-4621-9718-d0a8aa2fd7d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:05.109178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:05.110187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:05.111063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:05.111063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:05.111063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:05.119415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 426/845] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:05.122274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3434428-382c-4621-9718-d0a8aa2fd7d5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:05.152780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb57ea-295e-4105-8f65-5075793b2ca6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:05.198593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9ab25-81b0-42a2-bc5d-033b494940c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:05.199781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9ab25-81b0-42a2-bc5d-033b494940c6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:05.200061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9ab25-81b0-42a2-bc5d-033b494940c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:05.200322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9ab25-81b0-42a2-bc5d-033b494940c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:05.265875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3434428-382c-4621-9718-d0a8aa2fd7d5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:05.276015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:05.288735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9ab25-81b0-42a2-bc5d-033b494940c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:05.290632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9ab25-81b0-42a2-bc5d-033b494940c6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:05.307806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acb7c5-1b97-4738-b8a2-1564aa412560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:05.309223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acb7c5-1b97-4738-b8a2-1564aa412560 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:05.309960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acb7c5-1b97-4738-b8a2-1564aa412560 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:05.310897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acb7c5-1b97-4738-b8a2-1564aa412560 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:05.312043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acb7c5-1b97-4738-b8a2-1564aa412560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:05.312548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acb7c5-1b97-4738-b8a2-1564aa412560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:05.314329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acb7c5-1b97-4738-b8a2-1564aa412560 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:05.333189 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 427/846] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:05.334125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acb7c5-1b97-4738-b8a2-1564aa412560 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:05.394064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:05.408343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 420/847] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:05 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 05:15:05.418110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9ab25-81b0-42a2-bc5d-033b494940c6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:05.446707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acb7c5-1b97-4738-b8a2-1564aa412560 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:05.486677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bafcc-42a0-4f82-998f-450a1fcdbbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:05.487449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bafcc-42a0-4f82-998f-450a1fcdbbe4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:05.487989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bafcc-42a0-4f82-998f-450a1fcdbbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:05.488264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bafcc-42a0-4f82-998f-450a1fcdbbe4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:05.496779 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bafcc-42a0-4f82-998f-450a1fcdbbe4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:05.498364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bafcc-42a0-4f82-998f-450a1fcdbbe4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-2a8bafcc-42a0-4f82-998f-450a1fcdbbe4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:05.512452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:05.513411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 05:15:05.514348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:05.518011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 428/848] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:15:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:05.519667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bafcc-42a0-4f82-998f-450a1fcdbbe4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:05.534329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:05.537124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9ab25-81b0-42a2-bc5d-033b494940c6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:05.574579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203198e-146b-4036-9bd4-e29281cc3264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:05.575197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203198e-146b-4036-9bd4-e29281cc3264 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:05.575480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203198e-146b-4036-9bd4-e29281cc3264 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:05.575748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203198e-146b-4036-9bd4-e29281cc3264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:05.586583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203198e-146b-4036-9bd4-e29281cc3264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:05.588070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203198e-146b-4036-9bd4-e29281cc3264 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c203198e-146b-4036-9bd4-e29281cc3264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:05.604017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:05.605607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:05.607113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:05.607113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:05.607113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:05.607113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 421/849] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:05.611699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203198e-146b-4036-9bd4-e29281cc3264 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:05.645391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bafcc-42a0-4f82-998f-450a1fcdbbe4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:05.661561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:05.687929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aaac3-a9b7-4f6a-9b9f-fc3dddcbd419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:05.688904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aaac3-a9b7-4f6a-9b9f-fc3dddcbd419 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:05.689384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aaac3-a9b7-4f6a-9b9f-fc3dddcbd419 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:05.689874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aaac3-a9b7-4f6a-9b9f-fc3dddcbd419 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:05.690895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aaac3-a9b7-4f6a-9b9f-fc3dddcbd419 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:05.691402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aaac3-a9b7-4f6a-9b9f-fc3dddcbd419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:05.693009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aaac3-a9b7-4f6a-9b9f-fc3dddcbd419 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:05.718021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 429/850] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Sep 21 05:15:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:05.720189 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aaac3-a9b7-4f6a-9b9f-fc3dddcbd419 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:05.721780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203198e-146b-4036-9bd4-e29281cc3264 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:05.757901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb25b7-e560-4b86-a53b-9956603e36a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:05.758636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb25b7-e560-4b86-a53b-9956603e36a4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:05.758923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb25b7-e560-4b86-a53b-9956603e36a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:05.759193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb25b7-e560-4b86-a53b-9956603e36a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:05.759838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb25b7-e560-4b86-a53b-9956603e36a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:05.760165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb25b7-e560-4b86-a53b-9956603e36a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:05.763262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb25b7-e560-4b86-a53b-9956603e36a4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:05.782693 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 422/851] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:05.787790 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb25b7-e560-4b86-a53b-9956603e36a4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:05.791410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:05.843302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aaac3-a9b7-4f6a-9b9f-fc3dddcbd419 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:05.887564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626004a-d4cc-4006-a705-7105742bd32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:05.888372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626004a-d4cc-4006-a705-7105742bd32b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:05.888789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626004a-d4cc-4006-a705-7105742bd32b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:05.889132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626004a-d4cc-4006-a705-7105742bd32b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:05.904858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb25b7-e560-4b86-a53b-9956603e36a4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:05.936158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:05.946726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a6f6c-291a-4601-8a17-f92fcd0a67eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:05.947665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a6f6c-291a-4601-8a17-f92fcd0a67eb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:05.948089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a6f6c-291a-4601-8a17-f92fcd0a67eb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:05.948535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a6f6c-291a-4601-8a17-f92fcd0a67eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:05.956922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a6f6c-291a-4601-8a17-f92fcd0a67eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:05.958767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a6f6c-291a-4601-8a17-f92fcd0a67eb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:05.972360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626004a-d4cc-4006-a705-7105742bd32b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c87a6f6c-291a-4601-8a17-f92fcd0a67eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:05.972815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:05.973867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:05.975019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:05.975019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:05.975019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:05.975019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:05.975019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:05.975019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:05.975019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:05.975019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:05.975561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626004a-d4cc-4006-a705-7105742bd32b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:05.977125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 423/852] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:05.988039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a6f6c-291a-4601-8a17-f92fcd0a67eb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:06.069454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:06.098355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 430/853] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:05 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 05:15:06.099968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2d4c5-431a-4577-b172-4d4545d1a0d6 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.100320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2d4c5-431a-4577-b172-4d4545d1a0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.100666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2d4c5-431a-4577-b172-4d4545d1a0d6 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.102155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 431/854] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:06.103892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2d4c5-431a-4577-b172-4d4545d1a0d6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:06.120024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a6f6c-291a-4601-8a17-f92fcd0a67eb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:06.155306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b24ec-e8e9-44c4-9b5a-0de464f4efd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:06.155910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b24ec-e8e9-44c4-9b5a-0de464f4efd8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.156190 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b24ec-e8e9-44c4-9b5a-0de464f4efd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.156461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b24ec-e8e9-44c4-9b5a-0de464f4efd8 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.157147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b24ec-e8e9-44c4-9b5a-0de464f4efd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:06.157454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b24ec-e8e9-44c4-9b5a-0de464f4efd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:06.159149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b24ec-e8e9-44c4-9b5a-0de464f4efd8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:06.171538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 424/855] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:06.173008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbce67-4f33-4269-b4cd-987ede9e4068 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.173436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbce67-4f33-4269-b4cd-987ede9e4068 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.173852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbce67-4f33-4269-b4cd-987ede9e4068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.179248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:06.191357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5ccbce67-4f33-4269-b4cd-987ede9e4068 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:06.192472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5ccbce67-4f33-4269-b4cd-987ede9e4068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:06.211043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2d4c5-431a-4577-b172-4d4545d1a0d6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:06.244500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27687c2b-3e58-44ad-b61c-84d0e52a52ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:06.245119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27687c2b-3e58-44ad-b61c-84d0e52a52ec None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.245394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27687c2b-3e58-44ad-b61c-84d0e52a52ec None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.245661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27687c2b-3e58-44ad-b61c-84d0e52a52ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.255395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27687c2b-3e58-44ad-b61c-84d0e52a52ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:06.256809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27687c2b-3e58-44ad-b61c-84d0e52a52ec None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:06.257342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 425/856] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:06.258664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75975-4090-4560-a18e-4506b6981c4b None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.258955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75975-4090-4560-a18e-4506b6981c4b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.259234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75975-4090-4560-a18e-4506b6981c4b None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.265466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 426/857] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:06.265466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f75975-4090-4560-a18e-4506b6981c4b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-27687c2b-3e58-44ad-b61c-84d0e52a52ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:06.271765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:06.272992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:06.273880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:06.273880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:06.273880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:06.278056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 432/858] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:06.279297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fdd41-0507-44d5-b4f3-6b871e38c716 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.279579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fdd41-0507-44d5-b4f3-6b871e38c716 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.279856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fdd41-0507-44d5-b4f3-6b871e38c716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.297023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-380fdd41-0507-44d5-b4f3-6b871e38c716 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:06.298219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-380fdd41-0507-44d5-b4f3-6b871e38c716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:06.333755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:06.370371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 433/859] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:06.370371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fdd41-0507-44d5-b4f3-6b871e38c716 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:06.389712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f75975-4090-4560-a18e-4506b6981c4b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:06.426464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1839398-68e3-47d0-b150-64725ec20010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:06.427145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1839398-68e3-47d0-b150-64725ec20010 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.427604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1839398-68e3-47d0-b150-64725ec20010 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.427883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1839398-68e3-47d0-b150-64725ec20010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.428601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1839398-68e3-47d0-b150-64725ec20010 None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d2297673f49b4ce8bf40ac5f750d989f) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:06.428906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1839398-68e3-47d0-b150-64725ec20010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:06.430482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1839398-68e3-47d0-b150-64725ec20010 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:06.446047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 427/860] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:06 2022] GET /identity/v3/role_assignments?user.id=d2297673f49b4ce8bf40ac5f750d989f&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:06.446227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:06.501192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fdd41-0507-44d5-b4f3-6b871e38c716 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:06.530491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1839398-68e3-47d0-b150-64725ec20010 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:06.540085 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5038c8-0996-4323-bc4c-6f453c69c813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:06.541031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5038c8-0996-4323-bc4c-6f453c69c813 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.541031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5038c8-0996-4323-bc4c-6f453c69c813 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.541282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5038c8-0996-4323-bc4c-6f453c69c813 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.541878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5038c8-0996-4323-bc4c-6f453c69c813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:06.542295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5038c8-0996-4323-bc4c-6f453c69c813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:06.543719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5038c8-0996-4323-bc4c-6f453c69c813 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:06.556317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 434/861] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:06.564606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5038c8-0996-4323-bc4c-6f453c69c813 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:06.665397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1839398-68e3-47d0-b150-64725ec20010 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:06.693429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:06.710238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff489da5-6932-4189-a825-2004f88dd7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:06.710795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff489da5-6932-4189-a825-2004f88dd7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.711074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff489da5-6932-4189-a825-2004f88dd7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.711352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff489da5-6932-4189-a825-2004f88dd7b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.711934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff489da5-6932-4189-a825-2004f88dd7b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:06.713312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff489da5-6932-4189-a825-2004f88dd7b3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:06.721895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5038c8-0996-4323-bc4c-6f453c69c813 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-ff489da5-6932-4189-a825-2004f88dd7b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:06.724260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:06.725420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 05:15:06.726424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:06.727540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 428/862] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:15:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:06.728358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff489da5-6932-4189-a825-2004f88dd7b3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:06.739438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:06.763249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e33e6-d8f5-4365-966a-24c874c0e457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:06.763883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e33e6-d8f5-4365-966a-24c874c0e457 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.764159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e33e6-d8f5-4365-966a-24c874c0e457 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.764431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e33e6-d8f5-4365-966a-24c874c0e457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.772546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e33e6-d8f5-4365-966a-24c874c0e457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:06.774434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e33e6-d8f5-4365-966a-24c874c0e457 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-aa0e33e6-d8f5-4365-966a-24c874c0e457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:06.787650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:06.788605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:06.789483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:06.789483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:06.789483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:06.789483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:06.789483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:06.789483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:06.789483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:06.789483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:06.789483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 435/863] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:06.790473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e33e6-d8f5-4365-966a-24c874c0e457 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:06.844228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff489da5-6932-4189-a825-2004f88dd7b3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:06.880939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf83e33-2ff4-4d21-8c15-8d75ebfc1c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:06.881672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf83e33-2ff4-4d21-8c15-8d75ebfc1c13 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.882024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf83e33-2ff4-4d21-8c15-8d75ebfc1c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.882422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf83e33-2ff4-4d21-8c15-8d75ebfc1c13 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.882996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf83e33-2ff4-4d21-8c15-8d75ebfc1c13 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:06.884472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf83e33-2ff4-4d21-8c15-8d75ebfc1c13 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-cdf83e33-2ff4-4d21-8c15-8d75ebfc1c13 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:06.894581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:06.895682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:06.897162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 429/864] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Sep 21 05:15:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:06.897929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf83e33-2ff4-4d21-8c15-8d75ebfc1c13 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:06.908864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e33e6-d8f5-4365-966a-24c874c0e457 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:06.949708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:06.955442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce0393-a91f-4907-9fd0-a12c315905d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:06.956056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce0393-a91f-4907-9fd0-a12c315905d0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:06.956337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce0393-a91f-4907-9fd0-a12c315905d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:06.956609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce0393-a91f-4907-9fd0-a12c315905d0 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:06.957232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce0393-a91f-4907-9fd0-a12c315905d0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:06.957803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce0393-a91f-4907-9fd0-a12c315905d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:06.958985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce0393-a91f-4907-9fd0-a12c315905d0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:06.965765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 436/865] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:15:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:06.967126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce0393-a91f-4907-9fd0-a12c315905d0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:07.011794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:07.015660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf83e33-2ff4-4d21-8c15-8d75ebfc1c13 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:07.056640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410d877-5359-458c-bdb6-e42229921847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:07.057252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410d877-5359-458c-bdb6-e42229921847 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:07.057530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410d877-5359-458c-bdb6-e42229921847 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:07.057800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410d877-5359-458c-bdb6-e42229921847 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:07.059101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410d877-5359-458c-bdb6-e42229921847 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:07.059411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410d877-5359-458c-bdb6-e42229921847 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:07.060832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410d877-5359-458c-bdb6-e42229921847 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:07.073390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 430/866] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:07.074208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410d877-5359-458c-bdb6-e42229921847 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:07.091330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce0393-a91f-4907-9fd0-a12c315905d0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:07.129274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bda70-60e4-4a8b-952f-e18b33159c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:07.130271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bda70-60e4-4a8b-952f-e18b33159c06 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:07.130584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bda70-60e4-4a8b-952f-e18b33159c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:07.130875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bda70-60e4-4a8b-952f-e18b33159c06 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:07.131501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bda70-60e4-4a8b-952f-e18b33159c06 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:07.131800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bda70-60e4-4a8b-952f-e18b33159c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:07.133142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bda70-60e4-4a8b-952f-e18b33159c06 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:07.142384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 437/867] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:07.142384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bda70-60e4-4a8b-952f-e18b33159c06 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:07.190756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410d877-5359-458c-bdb6-e42229921847 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:07.199239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:07.229351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa7199-5836-476a-bdc7-b8d7fb8c2501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:07.229928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7199-5836-476a-bdc7-b8d7fb8c2501 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:07.230730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7199-5836-476a-bdc7-b8d7fb8c2501 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:07.231049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7199-5836-476a-bdc7-b8d7fb8c2501 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:07.231677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa7199-5836-476a-bdc7-b8d7fb8c2501 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:07.231977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa7199-5836-476a-bdc7-b8d7fb8c2501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:07.233343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa7199-5836-476a-bdc7-b8d7fb8c2501 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:07.238171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 431/868] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:15:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:07.238955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa7199-5836-476a-bdc7-b8d7fb8c2501 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:07.260763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:07.264858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bda70-60e4-4a8b-952f-e18b33159c06 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:07.313465 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b41c87-fd7c-4ac3-ada6-6a2aed360e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:07.314986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b41c87-fd7c-4ac3-ada6-6a2aed360e6f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:07.316253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b41c87-fd7c-4ac3-ada6-6a2aed360e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:07.316780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b41c87-fd7c-4ac3-ada6-6a2aed360e6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:07.318149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b41c87-fd7c-4ac3-ada6-6a2aed360e6f None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=ee0fe35d9e304ee495998c93c6909fa8) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:07.318685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b41c87-fd7c-4ac3-ada6-6a2aed360e6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:07.320560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b41c87-fd7c-4ac3-ada6-6a2aed360e6f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:07.328636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 438/869] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:07 2022] GET /identity/v3/role_assignments?user.id=ee0fe35d9e304ee495998c93c6909fa8&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:07.387535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa7199-5836-476a-bdc7-b8d7fb8c2501 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:07.433770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df958f-4ab4-44a6-a8a4-3800db39e18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:07.434442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df958f-4ab4-44a6-a8a4-3800db39e18a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:07.434698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df958f-4ab4-44a6-a8a4-3800db39e18a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:07.434970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df958f-4ab4-44a6-a8a4-3800db39e18a None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:07.435633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df958f-4ab4-44a6-a8a4-3800db39e18a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:07.435941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df958f-4ab4-44a6-a8a4-3800db39e18a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:07.437435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df958f-4ab4-44a6-a8a4-3800db39e18a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:07.444845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 432/870] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:08.785756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcedb9-2e30-459a-8eaa-2183d55b55e5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:08.786344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcedb9-2e30-459a-8eaa-2183d55b55e5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:08.786344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcedb9-2e30-459a-8eaa-2183d55b55e5 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:08.787907 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 439/871] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:08.793736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17a831-7da5-465a-8d00-d123e21dbdd1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:08.794162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17a831-7da5-465a-8d00-d123e21dbdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:08.794332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17a831-7da5-465a-8d00-d123e21dbdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:08.813336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7e17a831-7da5-465a-8d00-d123e21dbdd1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:08.814815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7e17a831-7da5-465a-8d00-d123e21dbdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:08.842319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:08.883997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 433/872] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:09.154165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcedb9-2e30-459a-8eaa-2183d55b55e5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:09.258245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcedb9-2e30-459a-8eaa-2183d55b55e5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:09.293414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac48f1-ed11-48b8-b1aa-8f95ebefefa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:09.294070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac48f1-ed11-48b8-b1aa-8f95ebefefa9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.294352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac48f1-ed11-48b8-b1aa-8f95ebefefa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.294626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac48f1-ed11-48b8-b1aa-8f95ebefefa9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.305478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac48f1-ed11-48b8-b1aa-8f95ebefefa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-27ac48f1-ed11-48b8-b1aa-8f95ebefefa9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:09.310280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:09.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:09.312118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 440/873] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Sep 21 05:15:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:09.317504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17a831-7da5-465a-8d00-d123e21dbdd1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:09.425327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17a831-7da5-465a-8d00-d123e21dbdd1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:09.460610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aab82e-721b-4c61-b204-528879eae5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:09.461221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aab82e-721b-4c61-b204-528879eae5fd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.461503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aab82e-721b-4c61-b204-528879eae5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.461773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aab82e-721b-4c61-b204-528879eae5fd None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.462454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aab82e-721b-4c61-b204-528879eae5fd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:09.462755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aab82e-721b-4c61-b204-528879eae5fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:09.464225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aab82e-721b-4c61-b204-528879eae5fd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:09.477287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 434/874] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Sep 21 05:15:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:09.487167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac48f1-ed11-48b8-b1aa-8f95ebefefa9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:09.517505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce48d07-56d6-416a-8c62-ea5cb68bc90e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.517834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce48d07-56d6-416a-8c62-ea5cb68bc90e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.518163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce48d07-56d6-416a-8c62-ea5cb68bc90e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.518617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:09.519543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 435/875] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:09.525450 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17c276-8fef-4ff8-94ee-9341433bfacf None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.525773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17c276-8fef-4ff8-94ee-9341433bfacf None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.526064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17c276-8fef-4ff8-94ee-9341433bfacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.545034 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5f17c276-8fef-4ff8-94ee-9341433bfacf None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:09.546285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f17c276-8fef-4ff8-94ee-9341433bfacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:09.573556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:09.613004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac48f1-ed11-48b8-b1aa-8f95ebefefa9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:09.641473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 436/876] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:09.643045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd2154-1c2a-4f36-b710-1915b0760b41 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.643522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd2154-1c2a-4f36-b710-1915b0760b41 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.643977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd2154-1c2a-4f36-b710-1915b0760b41 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.645596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 437/877] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:09.649775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66c584-802a-40c2-bb24-207f4c5da856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:09.650081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c584-802a-40c2-bb24-207f4c5da856 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.650343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c584-802a-40c2-bb24-207f4c5da856 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.650657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c584-802a-40c2-bb24-207f4c5da856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.651898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a47aec-891c-4713-b323-9ec598b16a52 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.652393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a47aec-891c-4713-b323-9ec598b16a52 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.652866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a47aec-891c-4713-b323-9ec598b16a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.662789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66c584-802a-40c2-bb24-207f4c5da856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:09.664217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66c584-802a-40c2-bb24-207f4c5da856 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:09.672360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-48a47aec-891c-4713-b323-9ec598b16a52 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:09.673828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-48a47aec-891c-4713-b323-9ec598b16a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-cd66c584-802a-40c2-bb24-207f4c5da856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:09.680445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:09.681374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:09.682815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:09.682815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:09.682815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:09.682815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 441/878] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:09.684070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fb765-3c21-4b06-bbea-85327df173c2 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.684354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fb765-3c21-4b06-bbea-85327df173c2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.685066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fb765-3c21-4b06-bbea-85327df173c2 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.686854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 442/879] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:09.688017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fb765-3c21-4b06-bbea-85327df173c2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:09.750604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 438/880] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:09.751832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b7179-6f38-4f03-9812-254631800249 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.752112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b7179-6f38-4f03-9812-254631800249 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.752382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b7179-6f38-4f03-9812-254631800249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.774707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:09.782954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d94b7179-6f38-4f03-9812-254631800249 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:09.784153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d94b7179-6f38-4f03-9812-254631800249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:09.801726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fb765-3c21-4b06-bbea-85327df173c2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:09.841958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f880e-e430-4ecd-8497-31066fdf4230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:09.842754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f880e-e430-4ecd-8497-31066fdf4230 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:09.843395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f880e-e430-4ecd-8497-31066fdf4230 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:09.843606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f880e-e430-4ecd-8497-31066fdf4230 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:09.844405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f880e-e430-4ecd-8497-31066fdf4230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:09.844760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f880e-e430-4ecd-8497-31066fdf4230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:09.846510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f880e-e430-4ecd-8497-31066fdf4230 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:09.861528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 443/881] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:09.866837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f880e-e430-4ecd-8497-31066fdf4230 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:09.877348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 439/882] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:09.923409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b7179-6f38-4f03-9812-254631800249 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:09.942052 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:09.984240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f880e-e430-4ecd-8497-31066fdf4230 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:10.023686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fd1b8-202c-4274-83c7-82ed8028d640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:10.024855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fd1b8-202c-4274-83c7-82ed8028d640 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:10.025329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fd1b8-202c-4274-83c7-82ed8028d640 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:10.025906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fd1b8-202c-4274-83c7-82ed8028d640 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:10.030132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:10.035396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fd1b8-202c-4274-83c7-82ed8028d640 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:10.037690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fd1b8-202c-4274-83c7-82ed8028d640 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:10.039845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b7179-6f38-4f03-9812-254631800249 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-9d7fd1b8-202c-4274-83c7-82ed8028d640 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:10.051701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:10.052674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 444/883] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:15:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:10.053593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fd1b8-202c-4274-83c7-82ed8028d640 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:10.082159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611057-e6e1-4dd4-838b-dd4a9b284d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:10.082995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611057-e6e1-4dd4-838b-dd4a9b284d00 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:10.083426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611057-e6e1-4dd4-838b-dd4a9b284d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:10.083859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611057-e6e1-4dd4-838b-dd4a9b284d00 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:10.084497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43611057-e6e1-4dd4-838b-dd4a9b284d00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:10.085836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43611057-e6e1-4dd4-838b-dd4a9b284d00 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:10.109982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 440/884] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:15:09 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:15:10.109982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611057-e6e1-4dd4-838b-dd4a9b284d00 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:10.194215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fd1b8-202c-4274-83c7-82ed8028d640 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:10.199513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:10.236097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611057-e6e1-4dd4-838b-dd4a9b284d00 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:10.239340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b36db2-e155-4793-bf02-517f4b1aa558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:10.239980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b36db2-e155-4793-bf02-517f4b1aa558 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:10.240255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b36db2-e155-4793-bf02-517f4b1aa558 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:10.240524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b36db2-e155-4793-bf02-517f4b1aa558 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:10.241096 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b36db2-e155-4793-bf02-517f4b1aa558 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:10.242535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b36db2-e155-4793-bf02-517f4b1aa558 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-69b36db2-e155-4793-bf02-517f4b1aa558 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:10.254079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:10.255205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:10.256071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:10.260664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 445/885] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:10.262122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b36db2-e155-4793-bf02-517f4b1aa558 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:10.280340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d998c-8db2-42c2-81a4-f8a00ffbce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:10.280991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d998c-8db2-42c2-81a4-f8a00ffbce88 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:10.281301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d998c-8db2-42c2-81a4-f8a00ffbce88 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:10.281596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d998c-8db2-42c2-81a4-f8a00ffbce88 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:10.282390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d998c-8db2-42c2-81a4-f8a00ffbce88 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:10.282701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d998c-8db2-42c2-81a4-f8a00ffbce88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:10.284739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d998c-8db2-42c2-81a4-f8a00ffbce88 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:10.302430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 441/886] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Sep 21 05:15:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:10.303361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d998c-8db2-42c2-81a4-f8a00ffbce88 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:10.331593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:10.379445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b36db2-e155-4793-bf02-517f4b1aa558 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:10.413229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d998c-8db2-42c2-81a4-f8a00ffbce88 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:10.416399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc2ab-edfa-4e2b-b0bd-0690f2c7f5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:10.417004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc2ab-edfa-4e2b-b0bd-0690f2c7f5e5 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:10.417312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc2ab-edfa-4e2b-b0bd-0690f2c7f5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:10.417586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc2ab-edfa-4e2b-b0bd-0690f2c7f5e5 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:10.418259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fc2ab-edfa-4e2b-b0bd-0690f2c7f5e5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:10.419611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fc2ab-edfa-4e2b-b0bd-0690f2c7f5e5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:10.436946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 446/887] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:15:10 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:10.438207 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc2ab-edfa-4e2b-b0bd-0690f2c7f5e5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:10.453530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:10.472866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400ee32-21d6-4854-969a-896de8cbef8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:10.474007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400ee32-21d6-4854-969a-896de8cbef8d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:10.474555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400ee32-21d6-4854-969a-896de8cbef8d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:10.475163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400ee32-21d6-4854-969a-896de8cbef8d None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:10.476242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400ee32-21d6-4854-969a-896de8cbef8d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:10.476788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400ee32-21d6-4854-969a-896de8cbef8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:10.478649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400ee32-21d6-4854-969a-896de8cbef8d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:10.490836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 442/888] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:10.492171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400ee32-21d6-4854-969a-896de8cbef8d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:10.576387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc2ab-edfa-4e2b-b0bd-0690f2c7f5e5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:10.596693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:10.618939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400ee32-21d6-4854-969a-896de8cbef8d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:10.620045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36bb1e-ee47-40ba-8d97-3aef366dc904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:10.620697 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36bb1e-ee47-40ba-8d97-3aef366dc904 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:10.620979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36bb1e-ee47-40ba-8d97-3aef366dc904 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:10.621253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36bb1e-ee47-40ba-8d97-3aef366dc904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:10.662585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f27801-9037-4b28-9aa9-532fd7cd6758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:10.663198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f27801-9037-4b28-9aa9-532fd7cd6758 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:10.663475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f27801-9037-4b28-9aa9-532fd7cd6758 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:10.663748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f27801-9037-4b28-9aa9-532fd7cd6758 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:10.664374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f27801-9037-4b28-9aa9-532fd7cd6758 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:10.664677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f27801-9037-4b28-9aa9-532fd7cd6758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:10.666080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f27801-9037-4b28-9aa9-532fd7cd6758 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:10.675568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 443/889] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:10.711690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36bb1e-ee47-40ba-8d97-3aef366dc904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:10.713576 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36bb1e-ee47-40ba-8d97-3aef366dc904 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:10.830513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 447/890] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:10 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:15:10.836613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f27801-9037-4b28-9aa9-532fd7cd6758 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:10.901229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:10.951899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f27801-9037-4b28-9aa9-532fd7cd6758 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:10.990750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146117-39e4-4cbb-8436-432ced4ab31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:10.991299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146117-39e4-4cbb-8436-432ced4ab31e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:10.991578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146117-39e4-4cbb-8436-432ced4ab31e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:10.991853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146117-39e4-4cbb-8436-432ced4ab31e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:11.003347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79146117-39e4-4cbb-8436-432ced4ab31e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:11.004821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79146117-39e4-4cbb-8436-432ced4ab31e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-79146117-39e4-4cbb-8436-432ced4ab31e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:11.022133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:11.023166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:11.024061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:11.024061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:11.024061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:11.024061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 444/891] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:11.029289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36bb1e-ee47-40ba-8d97-3aef366dc904 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:11.058960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:11.133264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36bb1e-ee47-40ba-8d97-3aef366dc904 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:11.167953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef0d5d-5cde-464e-a05c-04d728e17549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:11.168565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef0d5d-5cde-464e-a05c-04d728e17549 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:11.168848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef0d5d-5cde-464e-a05c-04d728e17549 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:11.169126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef0d5d-5cde-464e-a05c-04d728e17549 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:11.169761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef0d5d-5cde-464e-a05c-04d728e17549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:11.170149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef0d5d-5cde-464e-a05c-04d728e17549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:11.171508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef0d5d-5cde-464e-a05c-04d728e17549 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:11.183928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 448/892] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:11.189561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146117-39e4-4cbb-8436-432ced4ab31e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:11.299081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146117-39e4-4cbb-8436-432ced4ab31e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:11.306705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:11.337914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e76ab-ab8b-4a05-9f1b-30d4ee483b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:11.338928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e76ab-ab8b-4a05-9f1b-30d4ee483b7e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:11.339446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e76ab-ab8b-4a05-9f1b-30d4ee483b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:11.339962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e76ab-ab8b-4a05-9f1b-30d4ee483b7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:11.347490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e76ab-ab8b-4a05-9f1b-30d4ee483b7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:11.349219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e76ab-ab8b-4a05-9f1b-30d4ee483b7e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-196e76ab-ab8b-4a05-9f1b-30d4ee483b7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:11.361474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:11.362441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:11.363610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:11.363610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:11.363610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:11.363610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:11.363610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:11.363610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:11.363610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:11.363610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:11.366048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 445/893] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:11.374590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef0d5d-5cde-464e-a05c-04d728e17549 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:11.454248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:11.477657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef0d5d-5cde-464e-a05c-04d728e17549 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:11.513105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deeccf-299e-4729-8597-9c2bedef8417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:11.513720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deeccf-299e-4729-8597-9c2bedef8417 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:11.514030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deeccf-299e-4729-8597-9c2bedef8417 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:11.514307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deeccf-299e-4729-8597-9c2bedef8417 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:11.514997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deeccf-299e-4729-8597-9c2bedef8417 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:11.515306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deeccf-299e-4729-8597-9c2bedef8417 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:11.516719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deeccf-299e-4729-8597-9c2bedef8417 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:11.529125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 449/894] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:11.534986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e76ab-ab8b-4a05-9f1b-30d4ee483b7e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:11.657571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e76ab-ab8b-4a05-9f1b-30d4ee483b7e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:11.696709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66ce49-bc62-43d3-b851-29c84f34a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:11.697714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66ce49-bc62-43d3-b851-29c84f34a956 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:11.698316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66ce49-bc62-43d3-b851-29c84f34a956 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:11.698925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66ce49-bc62-43d3-b851-29c84f34a956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:11.699859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66ce49-bc62-43d3-b851-29c84f34a956 None admin] RBAC: Adding query filter params (role.id=688834a9cd994fa08909c0296f33b5a0, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d2297673f49b4ce8bf40ac5f750d989f) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:11.700374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66ce49-bc62-43d3-b851-29c84f34a956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:11.702012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66ce49-bc62-43d3-b851-29c84f34a956 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:11.708267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 446/895] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:11 2022] GET /identity/v3/role_assignments?user.id=d2297673f49b4ce8bf40ac5f750d989f&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=688834a9cd994fa08909c0296f33b5a0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:12.307174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b466b-469d-4c37-b170-596f2c71e018 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:12.307692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b466b-469d-4c37-b170-596f2c71e018 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:12.307796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b466b-469d-4c37-b170-596f2c71e018 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:12.309235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 450/896] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:12.314596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf453b-bcdd-4650-adb2-79fc8c301d35 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:12.315383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf453b-bcdd-4650-adb2-79fc8c301d35 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:12.315870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf453b-bcdd-4650-adb2-79fc8c301d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:12.338527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-22cf453b-bcdd-4650-adb2-79fc8c301d35 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:12.340275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-22cf453b-bcdd-4650-adb2-79fc8c301d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:12.365720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:12.413750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 447/897] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:12.691816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b466b-469d-4c37-b170-596f2c71e018 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:12.708425 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce7811-f281-4774-bbdb-c3c225baf210 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:12.708800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce7811-f281-4774-bbdb-c3c225baf210 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:12.708989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce7811-f281-4774-bbdb-c3c225baf210 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:12.710577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 448/898] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:12.715821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a60fa-d04f-453e-b94f-c307cbf6e8a2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:12.716112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a60fa-d04f-453e-b94f-c307cbf6e8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:12.716375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a60fa-d04f-453e-b94f-c307cbf6e8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:12.735589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d13a60fa-d04f-453e-b94f-c307cbf6e8a2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:12.736881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d13a60fa-d04f-453e-b94f-c307cbf6e8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:12.805925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b466b-469d-4c37-b170-596f2c71e018 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:12.815100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 449/899] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:12.844079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b2807-d634-4c1c-9e3e-6bd1bd651032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:12.844683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b2807-d634-4c1c-9e3e-6bd1bd651032 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:12.844966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b2807-d634-4c1c-9e3e-6bd1bd651032 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:12.845241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b2807-d634-4c1c-9e3e-6bd1bd651032 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:12.845815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b2807-d634-4c1c-9e3e-6bd1bd651032 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:12.847273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b2807-d634-4c1c-9e3e-6bd1bd651032 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-948b2807-d634-4c1c-9e3e-6bd1bd651032 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:12.856982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:12.857964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:12.858795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 451/900] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:15:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:12.865735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a60fa-d04f-453e-b94f-c307cbf6e8a2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:12.963962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:12.974267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a60fa-d04f-453e-b94f-c307cbf6e8a2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:13.011434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c1fb-1e16-4adf-a488-e2dfeb4c2ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:13.012040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c1fb-1e16-4adf-a488-e2dfeb4c2ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.012320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c1fb-1e16-4adf-a488-e2dfeb4c2ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.012592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c1fb-1e16-4adf-a488-e2dfeb4c2ad5 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.013212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1c1fb-1e16-4adf-a488-e2dfeb4c2ad5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:13.013512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1c1fb-1e16-4adf-a488-e2dfeb4c2ad5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:13.014940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1c1fb-1e16-4adf-a488-e2dfeb4c2ad5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:13.019934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 450/901] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:15:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:13.025732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c365bb-f639-4a76-8b8b-5818d0279d11 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.027321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c365bb-f639-4a76-8b8b-5818d0279d11 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.027321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c365bb-f639-4a76-8b8b-5818d0279d11 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.027644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c1fb-1e16-4adf-a488-e2dfeb4c2ad5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:13.027897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 452/902] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:13.034129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115adf-e9d6-4c1e-be15-7c81a596366e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.034483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115adf-e9d6-4c1e-be15-7c81a596366e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.034764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115adf-e9d6-4c1e-be15-7c81a596366e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.057671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-20115adf-e9d6-4c1e-be15-7c81a596366e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:13.060095 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-20115adf-e9d6-4c1e-be15-7c81a596366e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:13.135335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c1fb-1e16-4adf-a488-e2dfeb4c2ad5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:13.139563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 453/903] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:13.143209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20115adf-e9d6-4c1e-be15-7c81a596366e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:13.169897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752613d0-0f67-4363-855d-9dd52a8a69dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:13.170525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752613d0-0f67-4363-855d-9dd52a8a69dd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.170797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752613d0-0f67-4363-855d-9dd52a8a69dd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.171065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752613d0-0f67-4363-855d-9dd52a8a69dd None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.171679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752613d0-0f67-4363-855d-9dd52a8a69dd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:13.171979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752613d0-0f67-4363-855d-9dd52a8a69dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:13.173325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752613d0-0f67-4363-855d-9dd52a8a69dd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:13.183582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 451/904] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:15:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:13.184987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752613d0-0f67-4363-855d-9dd52a8a69dd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:13.203134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:13.261707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20115adf-e9d6-4c1e-be15-7c81a596366e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:13.299135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:13.305374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e975f-0a7c-4867-8970-905764944344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:13.306330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e975f-0a7c-4867-8970-905764944344 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.306785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752613d0-0f67-4363-855d-9dd52a8a69dd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:13.307197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e975f-0a7c-4867-8970-905764944344 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.308289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e975f-0a7c-4867-8970-905764944344 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.309226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e975f-0a7c-4867-8970-905764944344 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:13.311646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e975f-0a7c-4867-8970-905764944344 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d45e975f-0a7c-4867-8970-905764944344 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:13.322407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:13.323370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:13.329318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 454/905] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Sep 21 05:15:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:13.332170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e975f-0a7c-4867-8970-905764944344 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:13.353358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41944f-18c5-4354-ac46-7f33a4054ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:13.354003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41944f-18c5-4354-ac46-7f33a4054ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.354324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41944f-18c5-4354-ac46-7f33a4054ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.354615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41944f-18c5-4354-ac46-7f33a4054ed9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.355895 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41944f-18c5-4354-ac46-7f33a4054ed9 None admin] RBAC: Adding query filter params (interface=public, service_id=08acd08e70eb47a184cd5dd780b57e82, region_id=RegionOne) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:13.356256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41944f-18c5-4354-ac46-7f33a4054ed9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:13.357697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41944f-18c5-4354-ac46-7f33a4054ed9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:13.370363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 452/906] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Sep 21 05:15:13 2022] GET /identity/v3/endpoints?service_id=08acd08e70eb47a184cd5dd780b57e82&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:13.371749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41944f-18c5-4354-ac46-7f33a4054ed9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:13.446984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e975f-0a7c-4867-8970-905764944344 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:13.460638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:13.483675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d9593-b5dd-4f9e-83ae-fae63a0fdbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:13.483889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41944f-18c5-4354-ac46-7f33a4054ed9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:13.484280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d9593-b5dd-4f9e-83ae-fae63a0fdbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.484558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d9593-b5dd-4f9e-83ae-fae63a0fdbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.484826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d9593-b5dd-4f9e-83ae-fae63a0fdbf5 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.485560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d9593-b5dd-4f9e-83ae-fae63a0fdbf5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:13.485915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d9593-b5dd-4f9e-83ae-fae63a0fdbf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:13.487303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d9593-b5dd-4f9e-83ae-fae63a0fdbf5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:13.498234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 455/907] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:13.499319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d9593-b5dd-4f9e-83ae-fae63a0fdbf5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:13.521054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534834cc-b755-4144-b709-a2112c61c8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:13.521834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534834cc-b755-4144-b709-a2112c61c8fe None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.522249 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534834cc-b755-4144-b709-a2112c61c8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.522949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534834cc-b755-4144-b709-a2112c61c8fe None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.523556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534834cc-b755-4144-b709-a2112c61c8fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:13.525083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534834cc-b755-4144-b709-a2112c61c8fe None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:13.532583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 453/908] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Sep 21 05:15:13 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:15:13.533487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534834cc-b755-4144-b709-a2112c61c8fe None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:13.585130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:13.640681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d9593-b5dd-4f9e-83ae-fae63a0fdbf5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:13.674098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534834cc-b755-4144-b709-a2112c61c8fe None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:13.678728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833eb1e-e16a-44c0-bad2-dac41fa053e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:13.679356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833eb1e-e16a-44c0-bad2-dac41fa053e7 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.679653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833eb1e-e16a-44c0-bad2-dac41fa053e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.679901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833eb1e-e16a-44c0-bad2-dac41fa053e7 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.680562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833eb1e-e16a-44c0-bad2-dac41fa053e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:13.681960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833eb1e-e16a-44c0-bad2-dac41fa053e7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:13.702737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 456/909] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:15:13 2022] POST /identity/v3/services => generated 241 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:13.731544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e99f74-ea6a-4c7c-80b1-aa42244ed959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:13.732457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e99f74-ea6a-4c7c-80b1-aa42244ed959 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.732881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e99f74-ea6a-4c7c-80b1-aa42244ed959 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.733299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e99f74-ea6a-4c7c-80b1-aa42244ed959 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.734179 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e99f74-ea6a-4c7c-80b1-aa42244ed959 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:13.734670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e99f74-ea6a-4c7c-80b1-aa42244ed959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:13.736240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e99f74-ea6a-4c7c-80b1-aa42244ed959 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:13.742816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 454/910] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:13.749574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833eb1e-e16a-44c0-bad2-dac41fa053e7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:13.880326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833eb1e-e16a-44c0-bad2-dac41fa053e7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:13.901059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:13.924829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde2a2e-e6e6-4b31-940c-e1b97018e890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:13.925326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe864b2-1ba4-4555-849b-a879ece7bbe9 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.925437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde2a2e-e6e6-4b31-940c-e1b97018e890 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.925615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe864b2-1ba4-4555-849b-a879ece7bbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.925722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde2a2e-e6e6-4b31-940c-e1b97018e890 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.925897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe864b2-1ba4-4555-849b-a879ece7bbe9 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.926104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde2a2e-e6e6-4b31-940c-e1b97018e890 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.926781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde2a2e-e6e6-4b31-940c-e1b97018e890 None admin] RBAC: Adding query filter params (interface=public, service_id=051dc9dd08184693986229d4f983252e, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:13.927102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde2a2e-e6e6-4b31-940c-e1b97018e890 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:13.927498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 455/911] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:13.928502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde2a2e-e6e6-4b31-940c-e1b97018e890 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:13.932526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d435-7c2b-4041-b82a-ac46d05fc526 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:13.932526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d435-7c2b-4041-b82a-ac46d05fc526 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:13.932873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d435-7c2b-4041-b82a-ac46d05fc526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:13.935513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 457/912] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Sep 21 05:15:13 2022] GET /identity/v3/endpoints?service_id=051dc9dd08184693986229d4f983252e&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:13.941845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde2a2e-e6e6-4b31-940c-e1b97018e890 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:13.948109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:13.956213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-efb4d435-7c2b-4041-b82a-ac46d05fc526 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:13.957433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-efb4d435-7c2b-4041-b82a-ac46d05fc526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:14.042495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 456/913] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:14.072259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde2a2e-e6e6-4b31-940c-e1b97018e890 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:14.107634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a52623-5c47-457b-a7d9-bf8fc1723c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:14.108488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a52623-5c47-457b-a7d9-bf8fc1723c0b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:14.108914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a52623-5c47-457b-a7d9-bf8fc1723c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:14.109330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a52623-5c47-457b-a7d9-bf8fc1723c0b None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:14.110131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a52623-5c47-457b-a7d9-bf8fc1723c0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:14.111724 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a52623-5c47-457b-a7d9-bf8fc1723c0b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:14.126111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 458/914] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Sep 21 05:15:13 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:15:14.314956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4d435-7c2b-4041-b82a-ac46d05fc526 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:14.419279 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4d435-7c2b-4041-b82a-ac46d05fc526 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:14.455209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba87e-7c33-4ea6-b14b-aceb379392c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:14.456084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba87e-7c33-4ea6-b14b-aceb379392c6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:14.456529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba87e-7c33-4ea6-b14b-aceb379392c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:14.456946 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba87e-7c33-4ea6-b14b-aceb379392c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:14.459782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:14.470264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ba87e-7c33-4ea6-b14b-aceb379392c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-fa8ba87e-7c33-4ea6-b14b-aceb379392c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:14.475225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:14.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:14.477367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:14.480208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 457/915] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Sep 21 05:15:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:14.489075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a52623-5c47-457b-a7d9-bf8fc1723c0b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:14.536692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:14.601968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a52623-5c47-457b-a7d9-bf8fc1723c0b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:14.635097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f85b63-3d95-4a28-80ca-0712f4dd8052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:14.635706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f85b63-3d95-4a28-80ca-0712f4dd8052 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:14.636016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f85b63-3d95-4a28-80ca-0712f4dd8052 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:14.636289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f85b63-3d95-4a28-80ca-0712f4dd8052 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:14.636942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f85b63-3d95-4a28-80ca-0712f4dd8052 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:14.637242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f85b63-3d95-4a28-80ca-0712f4dd8052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:14.638631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f85b63-3d95-4a28-80ca-0712f4dd8052 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:14.647351 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 459/916] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Sep 21 05:15:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:14.652651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba87e-7c33-4ea6-b14b-aceb379392c6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:14.766333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba87e-7c33-4ea6-b14b-aceb379392c6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:14.802690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a9a78-7d2a-4668-a799-15b51b046ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:14.803320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a9a78-7d2a-4668-a799-15b51b046ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:14.803598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a9a78-7d2a-4668-a799-15b51b046ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:14.803868 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a9a78-7d2a-4668-a799-15b51b046ce9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:14.814320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a9a78-7d2a-4668-a799-15b51b046ce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:14.815774 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a9a78-7d2a-4668-a799-15b51b046ce9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-b68a9a78-7d2a-4668-a799-15b51b046ce9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:14.830737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:14.831708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:14.832604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:14.832604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:14.832604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:14.832604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 458/917] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:14.837878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f85b63-3d95-4a28-80ca-0712f4dd8052 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:14.943605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f85b63-3d95-4a28-80ca-0712f4dd8052 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:14.952815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:14.978772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25782f-1f4f-4c42-8a91-c923a64de3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:14.979383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25782f-1f4f-4c42-8a91-c923a64de3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:14.980230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25782f-1f4f-4c42-8a91-c923a64de3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:14.980509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25782f-1f4f-4c42-8a91-c923a64de3b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:14.981151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25782f-1f4f-4c42-8a91-c923a64de3b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:14.981453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25782f-1f4f-4c42-8a91-c923a64de3b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:14.982846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25782f-1f4f-4c42-8a91-c923a64de3b0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:14.995361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 460/918] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:15.000742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a9a78-7d2a-4668-a799-15b51b046ce9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:15.051271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:15.107381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a9a78-7d2a-4668-a799-15b51b046ce9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:15.142753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3fa63-85e4-4edd-8ad4-7d2a8c11c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:15.143381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3fa63-85e4-4edd-8ad4-7d2a8c11c3cf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:15.143660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3fa63-85e4-4edd-8ad4-7d2a8c11c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:15.143935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3fa63-85e4-4edd-8ad4-7d2a8c11c3cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:15.152165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3fa63-85e4-4edd-8ad4-7d2a8c11c3cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:15.153599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3fa63-85e4-4edd-8ad4-7d2a8c11c3cf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3ac3fa63-85e4-4edd-8ad4-7d2a8c11c3cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:15.169046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:15.170015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:15.170952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 459/919] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:15:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:15.176046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25782f-1f4f-4c42-8a91-c923a64de3b0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:15.281536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25782f-1f4f-4c42-8a91-c923a64de3b0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:15.315787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dd985-75cc-4ba4-92bc-aa0e1a9dc8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:15.316398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dd985-75cc-4ba4-92bc-aa0e1a9dc8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:15.316681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dd985-75cc-4ba4-92bc-aa0e1a9dc8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:15.316959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dd985-75cc-4ba4-92bc-aa0e1a9dc8f1 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:15.317678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776dd985-75cc-4ba4-92bc-aa0e1a9dc8f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:15.318008 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776dd985-75cc-4ba4-92bc-aa0e1a9dc8f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:15.319432 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776dd985-75cc-4ba4-92bc-aa0e1a9dc8f1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:15.338076 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 461/920] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Sep 21 05:15:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:15.344018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3fa63-85e4-4edd-8ad4-7d2a8c11c3cf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:15.448132 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3fa63-85e4-4edd-8ad4-7d2a8c11c3cf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:15.465961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:15.489707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3b9a1-92df-48ad-af92-29dbee34785c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:15.490430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3b9a1-92df-48ad-af92-29dbee34785c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:15.490717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3b9a1-92df-48ad-af92-29dbee34785c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:15.490991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3b9a1-92df-48ad-af92-29dbee34785c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:15.503889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3b9a1-92df-48ad-af92-29dbee34785c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:15.505412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3b9a1-92df-48ad-af92-29dbee34785c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-74b3b9a1-92df-48ad-af92-29dbee34785c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:15.520756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:15.522959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:15.525018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:15.525018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:15.525018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:15.525018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 460/921] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:15.528084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dd985-75cc-4ba4-92bc-aa0e1a9dc8f1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:15.556179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:15.634169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dd985-75cc-4ba4-92bc-aa0e1a9dc8f1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:15.669138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2459c-1013-4355-8b73-eaec871e41f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:15.669750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2459c-1013-4355-8b73-eaec871e41f6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:15.670059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2459c-1013-4355-8b73-eaec871e41f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:15.670332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2459c-1013-4355-8b73-eaec871e41f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:15.670967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2459c-1013-4355-8b73-eaec871e41f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:15.671272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2459c-1013-4355-8b73-eaec871e41f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:15.672675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2459c-1013-4355-8b73-eaec871e41f6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:15.685372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 462/922] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:15.690601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3b9a1-92df-48ad-af92-29dbee34785c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:15.804478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3b9a1-92df-48ad-af92-29dbee34785c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:15.840244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22933d9e-5a65-40ab-992c-3896e16dc294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:15.840878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22933d9e-5a65-40ab-992c-3896e16dc294 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:15.841155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22933d9e-5a65-40ab-992c-3896e16dc294 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:15.841429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22933d9e-5a65-40ab-992c-3896e16dc294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:15.848351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22933d9e-5a65-40ab-992c-3896e16dc294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:15.849804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22933d9e-5a65-40ab-992c-3896e16dc294 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-22933d9e-5a65-40ab-992c-3896e16dc294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:15.861624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:15.862813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:15.863677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:15.863677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:15.863677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:15.863677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:15.863677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:15.863677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:15.863677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:15.863677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:15.863677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 461/923] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:15.868772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2459c-1013-4355-8b73-eaec871e41f6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:15.892632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70c98d-af56-4e3d-9333-625c038a4af9 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:15.892926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70c98d-af56-4e3d-9333-625c038a4af9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:15.893189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70c98d-af56-4e3d-9333-625c038a4af9 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:15.898202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 462/924] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:15.900325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d859c-76b4-48da-b37e-0b82c10d9d61 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:15.900604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d859c-76b4-48da-b37e-0b82c10d9d61 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:15.900872 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d859c-76b4-48da-b37e-0b82c10d9d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:15.919584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5d8d859c-76b4-48da-b37e-0b82c10d9d61 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:15.920782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5d8d859c-76b4-48da-b37e-0b82c10d9d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:15.936251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:15.974992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:15.997582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 463/925] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:15.998684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2459c-1013-4355-8b73-eaec871e41f6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:16.041336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e0b2a-5e86-427e-ae0d-62ea7873f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:16.042567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e0b2a-5e86-427e-ae0d-62ea7873f014 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.042865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e0b2a-5e86-427e-ae0d-62ea7873f014 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.043144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e0b2a-5e86-427e-ae0d-62ea7873f014 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.043842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e0b2a-5e86-427e-ae0d-62ea7873f014 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:16.044152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e0b2a-5e86-427e-ae0d-62ea7873f014 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:16.045690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e0b2a-5e86-427e-ae0d-62ea7873f014 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:16.059010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 463/926] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:16.064704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d859c-76b4-48da-b37e-0b82c10d9d61 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:16.126479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6e5ff-e4a2-41c3-84e7-7b05ab414d7a None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.126674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6e5ff-e4a2-41c3-84e7-7b05ab414d7a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.126938 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6e5ff-e4a2-41c3-84e7-7b05ab414d7a None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.129621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 464/927] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:16.133664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2fd40-b511-492d-972b-c14290a0e191 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.133959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2fd40-b511-492d-972b-c14290a0e191 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.134238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2fd40-b511-492d-972b-c14290a0e191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.152275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1ec2fd40-b511-492d-972b-c14290a0e191 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:16.153362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1ec2fd40-b511-492d-972b-c14290a0e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:16.178540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d859c-76b4-48da-b37e-0b82c10d9d61 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:16.215646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21779a-1da1-4ba3-86e2-28c67a817292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:16.216304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21779a-1da1-4ba3-86e2-28c67a817292 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.216579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21779a-1da1-4ba3-86e2-28c67a817292 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.216844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21779a-1da1-4ba3-86e2-28c67a817292 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d2297673f49b4ce8bf40ac5f750d989f/roles/688834a9cd994fa08909c0296f33b5a0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.219001 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21779a-1da1-4ba3-86e2-28c67a817292 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=d2297673f49b4ce8bf40ac5f750d989f, role_id=688834a9cd994fa08909c0296f33b5a0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:16.220455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21779a-1da1-4ba3-86e2-28c67a817292 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:16.241974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 464/928] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:15:16 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d2297673f49b4ce8bf40ac5f750d989f/roles/688834a9cd994fa08909c0296f33b5a0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:15:16.250153 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 465/929] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:16.289310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21779a-1da1-4ba3-86e2-28c67a817292 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:16.305450 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:16.410072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21779a-1da1-4ba3-86e2-28c67a817292 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:16.419295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5251eb9-78d9-4006-bef4-5be69da6f2d0 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.419586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5251eb9-78d9-4006-bef4-5be69da6f2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.419850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5251eb9-78d9-4006-bef4-5be69da6f2d0 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.421428 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 466/930] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:16.426575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a03dc-f1cd-41c1-ae2a-25c1141e9bd2 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.427111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a03dc-f1cd-41c1-ae2a-25c1141e9bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.427449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a03dc-f1cd-41c1-ae2a-25c1141e9bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.447265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814885e-f5b0-4e6c-bcba-9798e04e5efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:16.447891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814885e-f5b0-4e6c-bcba-9798e04e5efe None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.448172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814885e-f5b0-4e6c-bcba-9798e04e5efe None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.448739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814885e-f5b0-4e6c-bcba-9798e04e5efe None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.449034 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814885e-f5b0-4e6c-bcba-9798e04e5efe None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:16.450802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814885e-f5b0-4e6c-bcba-9798e04e5efe None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:16.450909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-203a03dc-f1cd-41c1-ae2a-25c1141e9bd2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:16.451927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-203a03dc-f1cd-41c1-ae2a-25c1141e9bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-4814885e-f5b0-4e6c-bcba-9798e04e5efe None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:16.461004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:16.462272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 05:15:16.463109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:16.465953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 465/931] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:15:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:16.472796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814885e-f5b0-4e6c-bcba-9798e04e5efe None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:16.531427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 467/932] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:16.534949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a03dc-f1cd-41c1-ae2a-25c1141e9bd2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:16.546372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:16.560246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:16.594401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814885e-f5b0-4e6c-bcba-9798e04e5efe None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:16.640646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca100a6-0ef4-4681-8599-2124fd1094dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:16.641255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca100a6-0ef4-4681-8599-2124fd1094dc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.641533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca100a6-0ef4-4681-8599-2124fd1094dc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.641801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca100a6-0ef4-4681-8599-2124fd1094dc None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.642513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca100a6-0ef4-4681-8599-2124fd1094dc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:16.642814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca100a6-0ef4-4681-8599-2124fd1094dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:16.644409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca100a6-0ef4-4681-8599-2124fd1094dc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:16.654111 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 466/933] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:15:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:16.656173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a03dc-f1cd-41c1-ae2a-25c1141e9bd2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:16.656311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca100a6-0ef4-4681-8599-2124fd1094dc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:16.693142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c18fda-54dd-4d40-8e57-573109362286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:16.693798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c18fda-54dd-4d40-8e57-573109362286 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.694109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c18fda-54dd-4d40-8e57-573109362286 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.694381 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c18fda-54dd-4d40-8e57-573109362286 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.695141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c18fda-54dd-4d40-8e57-573109362286 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:16.697472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c18fda-54dd-4d40-8e57-573109362286 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d7c18fda-54dd-4d40-8e57-573109362286 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:16.707901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:16.708890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:16.709717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 468/934] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:16.716895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c18fda-54dd-4d40-8e57-573109362286 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:16.763858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca100a6-0ef4-4681-8599-2124fd1094dc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:16.792316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:16.802341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31694496-7133-4b35-b834-d3187961f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:16.802947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31694496-7133-4b35-b834-d3187961f248 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.803221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31694496-7133-4b35-b834-d3187961f248 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.803491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31694496-7133-4b35-b834-d3187961f248 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.804205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31694496-7133-4b35-b834-d3187961f248 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:16.804525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31694496-7133-4b35-b834-d3187961f248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:16.806103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31694496-7133-4b35-b834-d3187961f248 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:16.821121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 467/935] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Sep 21 05:15:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:16.824195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c18fda-54dd-4d40-8e57-573109362286 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:16.824950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31694496-7133-4b35-b834-d3187961f248 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:16.829199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:16.860759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98c13-e120-42de-808b-1f9d570ba972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:16.861478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98c13-e120-42de-808b-1f9d570ba972 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.861764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98c13-e120-42de-808b-1f9d570ba972 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.862074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98c13-e120-42de-808b-1f9d570ba972 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.862958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98c13-e120-42de-808b-1f9d570ba972 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:16.863263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98c13-e120-42de-808b-1f9d570ba972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:16.864618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98c13-e120-42de-808b-1f9d570ba972 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:16.869580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 469/936] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:16.870207 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98c13-e120-42de-808b-1f9d570ba972 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:16.936318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31694496-7133-4b35-b834-d3187961f248 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:16.971285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f4e3e-10ad-46ae-a73d-b10bf565a2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:16.972162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f4e3e-10ad-46ae-a73d-b10bf565a2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:16.972589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f4e3e-10ad-46ae-a73d-b10bf565a2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:16.972867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f4e3e-10ad-46ae-a73d-b10bf565a2b7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:16.973446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f4e3e-10ad-46ae-a73d-b10bf565a2b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:16.974838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f4e3e-10ad-46ae-a73d-b10bf565a2b7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:16.977781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98c13-e120-42de-808b-1f9d570ba972 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-204f4e3e-10ad-46ae-a73d-b10bf565a2b7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:16.983625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:16.984556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 05:15:16.985432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:16.989865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 468/937] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Sep 21 05:15:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:16.990602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f4e3e-10ad-46ae-a73d-b10bf565a2b7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:17.017827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed76ceb-0c45-4e0e-b81e-ce91c698a026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:17.018776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed76ceb-0c45-4e0e-b81e-ce91c698a026 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:17.019052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed76ceb-0c45-4e0e-b81e-ce91c698a026 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:17.019370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed76ceb-0c45-4e0e-b81e-ce91c698a026 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:17.019975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed76ceb-0c45-4e0e-b81e-ce91c698a026 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:17.021310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed76ceb-0c45-4e0e-b81e-ce91c698a026 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:17.026056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:17.053986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 470/938] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:15:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:17.053986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed76ceb-0c45-4e0e-b81e-ce91c698a026 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:17.067226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:17.114336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f4e3e-10ad-46ae-a73d-b10bf565a2b7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:17.168479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653490ce-656d-4362-8421-eb0d61a3c998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:17.169220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653490ce-656d-4362-8421-eb0d61a3c998 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:17.169566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653490ce-656d-4362-8421-eb0d61a3c998 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:17.170007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653490ce-656d-4362-8421-eb0d61a3c998 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:17.170731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653490ce-656d-4362-8421-eb0d61a3c998 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:17.171149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653490ce-656d-4362-8421-eb0d61a3c998 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:17.173586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653490ce-656d-4362-8421-eb0d61a3c998 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:17.185634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 469/939] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:17.187508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653490ce-656d-4362-8421-eb0d61a3c998 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:17.201490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed76ceb-0c45-4e0e-b81e-ce91c698a026 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:17.242322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da970a-7b25-4697-b67c-dfd0ec26bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:17.242870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da970a-7b25-4697-b67c-dfd0ec26bd0c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:17.243170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da970a-7b25-4697-b67c-dfd0ec26bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:17.243462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da970a-7b25-4697-b67c-dfd0ec26bd0c None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:17.244129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da970a-7b25-4697-b67c-dfd0ec26bd0c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:17.244434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da970a-7b25-4697-b67c-dfd0ec26bd0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:17.246548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da970a-7b25-4697-b67c-dfd0ec26bd0c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:17.252632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 471/940] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:17.257781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da970a-7b25-4697-b67c-dfd0ec26bd0c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:17.313855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653490ce-656d-4362-8421-eb0d61a3c998 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:17.325314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:17.345616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:17.364440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a253c7-7c8d-4f68-931d-b82f3c31760e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:17.365201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a253c7-7c8d-4f68-931d-b82f3c31760e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:17.365596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a253c7-7c8d-4f68-931d-b82f3c31760e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:17.366006 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a253c7-7c8d-4f68-931d-b82f3c31760e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:17.366806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a253c7-7c8d-4f68-931d-b82f3c31760e None admin] RBAC: Adding query filter params (interface=public, service_id=5f8fb93ee1074f9c88e2a7d2b87bf591, region_id=RegionOne) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:17.367215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a253c7-7c8d-4f68-931d-b82f3c31760e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:17.369514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a253c7-7c8d-4f68-931d-b82f3c31760e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:17.377613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 470/941] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Sep 21 05:15:17 2022] GET /identity/v3/endpoints?service_id=5f8fb93ee1074f9c88e2a7d2b87bf591&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:17.382160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da970a-7b25-4697-b67c-dfd0ec26bd0c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:17.383037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a253c7-7c8d-4f68-931d-b82f3c31760e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:17.428831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2c369-e1a7-4c7f-b7ba-2eed7d130843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:17.429623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2c369-e1a7-4c7f-b7ba-2eed7d130843 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:17.430042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2c369-e1a7-4c7f-b7ba-2eed7d130843 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:17.430417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2c369-e1a7-4c7f-b7ba-2eed7d130843 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:17.431603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2c369-e1a7-4c7f-b7ba-2eed7d130843 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:17.432060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2c369-e1a7-4c7f-b7ba-2eed7d130843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:17.434667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2c369-e1a7-4c7f-b7ba-2eed7d130843 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:17.446066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 472/942] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:17.447314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2c369-e1a7-4c7f-b7ba-2eed7d130843 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:17.508620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a253c7-7c8d-4f68-931d-b82f3c31760e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:17.552540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481639e2-47d3-4f44-8f22-ffb09c4931e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:17.553556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481639e2-47d3-4f44-8f22-ffb09c4931e5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:17.554062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481639e2-47d3-4f44-8f22-ffb09c4931e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:17.554553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481639e2-47d3-4f44-8f22-ffb09c4931e5 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:17.555354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481639e2-47d3-4f44-8f22-ffb09c4931e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:17.556990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481639e2-47d3-4f44-8f22-ffb09c4931e5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:17.564212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 471/943] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Sep 21 05:15:17 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:15:17.574032 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2c369-e1a7-4c7f-b7ba-2eed7d130843 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:17.597986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:17.617658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b04b-031c-4fd4-8ce7-2d03c85438fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:17.618323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b04b-031c-4fd4-8ce7-2d03c85438fb None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:17.618601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b04b-031c-4fd4-8ce7-2d03c85438fb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:17.618872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b04b-031c-4fd4-8ce7-2d03c85438fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:17.619562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b04b-031c-4fd4-8ce7-2d03c85438fb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:17.621568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b04b-031c-4fd4-8ce7-2d03c85438fb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:17.648826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 473/944] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:15:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:18.521424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be885b4-1cde-40a8-96dc-d4177934d218 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:18.521968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be885b4-1cde-40a8-96dc-d4177934d218 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:18.521968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be885b4-1cde-40a8-96dc-d4177934d218 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:18.523428 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 472/945] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:18.528868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a96b5a-8977-4151-8080-48ea4d19f76d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:18.529377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a96b5a-8977-4151-8080-48ea4d19f76d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:18.529845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a96b5a-8977-4151-8080-48ea4d19f76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:18.548848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-81a96b5a-8977-4151-8080-48ea4d19f76d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:18.550099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-81a96b5a-8977-4151-8080-48ea4d19f76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:18.631455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 474/946] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:18.896757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be885b4-1cde-40a8-96dc-d4177934d218 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:18.955839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:19.006176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be885b4-1cde-40a8-96dc-d4177934d218 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:19.040374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3e5e1-8f63-4fa0-8f3c-a0ee12d4c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:19.040999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3e5e1-8f63-4fa0-8f3c-a0ee12d4c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.041331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3e5e1-8f63-4fa0-8f3c-a0ee12d4c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.041607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3e5e1-8f63-4fa0-8f3c-a0ee12d4c8f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.053461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3e5e1-8f63-4fa0-8f3c-a0ee12d4c8f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-71a3e5e1-8f63-4fa0-8f3c-a0ee12d4c8f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:19.058735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:19.059662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:19.060536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 473/947] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Sep 21 05:15:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:19.066134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a96b5a-8977-4151-8080-48ea4d19f76d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:19.163116 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:19.166291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a96b5a-8977-4151-8080-48ea4d19f76d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:19.206794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ac869-08ca-42e3-b87f-359849f464ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:19.207770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ac869-08ca-42e3-b87f-359849f464ec None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.208272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ac869-08ca-42e3-b87f-359849f464ec None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.208745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ac869-08ca-42e3-b87f-359849f464ec None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.209615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ac869-08ca-42e3-b87f-359849f464ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:19.210156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ac869-08ca-42e3-b87f-359849f464ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:19.211722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ac869-08ca-42e3-b87f-359849f464ec None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:19.221298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 475/948] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Sep 21 05:15:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:19.226739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3e5e1-8f63-4fa0-8f3c-a0ee12d4c8f6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:19.334388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3e5e1-8f63-4fa0-8f3c-a0ee12d4c8f6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:19.352647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:19.378221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481a6c5-98b3-4dee-bcd6-4adfb8bce3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:19.378851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481a6c5-98b3-4dee-bcd6-4adfb8bce3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.379126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481a6c5-98b3-4dee-bcd6-4adfb8bce3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.379394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481a6c5-98b3-4dee-bcd6-4adfb8bce3c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.390972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481a6c5-98b3-4dee-bcd6-4adfb8bce3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:19.392412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481a6c5-98b3-4dee-bcd6-4adfb8bce3c8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-1481a6c5-98b3-4dee-bcd6-4adfb8bce3c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:19.407521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:19.408502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:19.409373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:19.409373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:19.409373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:19.409373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 474/949] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:19.414461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ac869-08ca-42e3-b87f-359849f464ec None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:19.426543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb45cd4-819b-4e47-88e4-7f9a856d72f4 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.426839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb45cd4-819b-4e47-88e4-7f9a856d72f4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.427121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb45cd4-819b-4e47-88e4-7f9a856d72f4 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.429079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 475/950] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:19.434605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e3b7a-1401-441a-b44e-e14d8eafa5b2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.434952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e3b7a-1401-441a-b44e-e14d8eafa5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.435139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e3b7a-1401-441a-b44e-e14d8eafa5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.453916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6c0e3b7a-1401-441a-b44e-e14d8eafa5b2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:19.455143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6c0e3b7a-1401-441a-b44e-e14d8eafa5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:19.533362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ac869-08ca-42e3-b87f-359849f464ec None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:19.542255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 476/951] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:19.570645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbd8a0-e68b-402c-9941-529249afcb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:19.571594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbd8a0-e68b-402c-9941-529249afcb43 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.572092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbd8a0-e68b-402c-9941-529249afcb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.572557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbd8a0-e68b-402c-9941-529249afcb43 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.573984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbd8a0-e68b-402c-9941-529249afcb43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:19.574559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbd8a0-e68b-402c-9941-529249afcb43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:19.576090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbd8a0-e68b-402c-9941-529249afcb43 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:19.601337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 476/952] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:19.607958 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e3b7a-1401-441a-b44e-e14d8eafa5b2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:19.711497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:19.718219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e3b7a-1401-441a-b44e-e14d8eafa5b2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:19.754227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31759f-1820-42c7-8b20-adf59ac01c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:19.755210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31759f-1820-42c7-8b20-adf59ac01c1f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.755683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31759f-1820-42c7-8b20-adf59ac01c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.756147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31759f-1820-42c7-8b20-adf59ac01c1f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.763787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31759f-1820-42c7-8b20-adf59ac01c1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:19.765486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31759f-1820-42c7-8b20-adf59ac01c1f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-1b31759f-1820-42c7-8b20-adf59ac01c1f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:19.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:19.778599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 05:15:19.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:19.781921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 477/953] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:15:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:19.787960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbd8a0-e68b-402c-9941-529249afcb43 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:19.799107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2135fb-2d70-4e9c-9b1d-43967ba4181f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.799397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2135fb-2d70-4e9c-9b1d-43967ba4181f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.799661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2135fb-2d70-4e9c-9b1d-43967ba4181f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.801099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 478/954] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:19.806461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363188c6-c57c-4f0c-b308-9423975fa6bd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.806743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363188c6-c57c-4f0c-b308-9423975fa6bd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.807013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363188c6-c57c-4f0c-b308-9423975fa6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.818273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:19.826226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-363188c6-c57c-4f0c-b308-9423975fa6bd None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:19.827460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-363188c6-c57c-4f0c-b308-9423975fa6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:19.901892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 479/955] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:19.902748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363188c6-c57c-4f0c-b308-9423975fa6bd None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:19.912298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbd8a0-e68b-402c-9941-529249afcb43 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:19.958113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66740001-4ff3-4cf1-a594-e0dd5209b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:19.959042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66740001-4ff3-4cf1-a594-e0dd5209b58b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.959325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66740001-4ff3-4cf1-a594-e0dd5209b58b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.959597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66740001-4ff3-4cf1-a594-e0dd5209b58b None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.960571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66740001-4ff3-4cf1-a594-e0dd5209b58b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:19.961331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66740001-4ff3-4cf1-a594-e0dd5209b58b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:19.963125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66740001-4ff3-4cf1-a594-e0dd5209b58b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:19.982066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 477/956] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Sep 21 05:15:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:19.983410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182c8fe-e219-487b-b8a4-b5f55d7cc186 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:19.983729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182c8fe-e219-487b-b8a4-b5f55d7cc186 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:19.984003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182c8fe-e219-487b-b8a4-b5f55d7cc186 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:19.989396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 478/957] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:19.989396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182c8fe-e219-487b-b8a4-b5f55d7cc186 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.002080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:20.017637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363188c6-c57c-4f0c-b308-9423975fa6bd None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.054740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11888480-449a-4757-a5c8-207fcee569f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:20.055344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11888480-449a-4757-a5c8-207fcee569f0 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.055622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11888480-449a-4757-a5c8-207fcee569f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.055895 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11888480-449a-4757-a5c8-207fcee569f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.056529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11888480-449a-4757-a5c8-207fcee569f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:20.058014 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11888480-449a-4757-a5c8-207fcee569f0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:20.060589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING keystone.server.flask.common [None req-11888480-449a-4757-a5c8-207fcee569f0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 21 05:15:20.078274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.sql.core [None req-11888480-449a-4757-a5c8-207fcee569f0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 21 05:15:20.078274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 21 05:15:20.078274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [parameters: {'id': 'cffe36c6247942489617071f6c7749fa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 21 05:15:20.078274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81411) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 21 05:15:20.078676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-11888480-449a-4757-a5c8-207fcee569f0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cffe36c6247942489617071f6c7749fa.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cffe36c6247942489617071f6c7749fa. Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 05:15:20.085760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 21 05:15:20.086737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application yield resource Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.rollback() Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.force_reraise() Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise self.value Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application session.commit() Sep 21 05:15:20.087694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._prepare_impl() Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.flush() Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._flush(objects) Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application compat.raise_( Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application flush_context.execute() Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 21 05:15:20.088568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rec.execute(self) Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 21 05:15:20.090687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 05:15:20.092413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [parameters: {'id': 'cffe36c6247942489617071f6c7749fa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.093547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:20.094663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cffe36c6247942489617071f6c7749fa. Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 480/958] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Sep 21 05:15:19 2022] POST /identity/v3/projects => generated 262 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cdf27-478d-41a4-b4d7-92e717cb167e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.095923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cdf27-478d-41a4-b4d7-92e717cb167e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.096767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cdf27-478d-41a4-b4d7-92e717cb167e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.109903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-789cdf27-478d-41a4-b4d7-92e717cb167e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:20.111473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-789cdf27-478d-41a4-b4d7-92e717cb167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:20.113158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182c8fe-e219-487b-b8a4-b5f55d7cc186 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.150551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae77d7-6496-4cb5-9777-5886c4929e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:20.151974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae77d7-6496-4cb5-9777-5886c4929e8e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.152398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae77d7-6496-4cb5-9777-5886c4929e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.152872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae77d7-6496-4cb5-9777-5886c4929e8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.181848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 481/959] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:20.189777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cdf27-478d-41a4-b4d7-92e717cb167e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.235576 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae77d7-6496-4cb5-9777-5886c4929e8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:20.236958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae77d7-6496-4cb5-9777-5886c4929e8e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:20.256502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:20.287837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cdf27-478d-41a4-b4d7-92e717cb167e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.324135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32c617-cee3-498e-9133-1628eb253a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:20.324791 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c617-cee3-498e-9133-1628eb253a0c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.325070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c617-cee3-498e-9133-1628eb253a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.325340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c617-cee3-498e-9133-1628eb253a0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.326243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:20.332244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32c617-cee3-498e-9133-1628eb253a0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:20.333877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32c617-cee3-498e-9133-1628eb253a0c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-ca32c617-cee3-498e-9133-1628eb253a0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:20.345403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:20.346491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:20.347437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:20.347437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:20.347437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:20.347437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:20.347437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:20.347437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:20.347437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:20.347437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.347437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 482/960] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:20.348000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32c617-cee3-498e-9133-1628eb253a0c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.355681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 479/961] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:19 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 05:15:20.357153 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae77d7-6496-4cb5-9777-5886c4929e8e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.451170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32c617-cee3-498e-9133-1628eb253a0c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.470999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae77d7-6496-4cb5-9777-5886c4929e8e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.484382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:20.490536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199181ef-a3e8-487d-87bc-f0a868b2a484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:20.491097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199181ef-a3e8-487d-87bc-f0a868b2a484 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.491375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199181ef-a3e8-487d-87bc-f0a868b2a484 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.491640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199181ef-a3e8-487d-87bc-f0a868b2a484 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.492240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199181ef-a3e8-487d-87bc-f0a868b2a484 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:20.493585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199181ef-a3e8-487d-87bc-f0a868b2a484 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-199181ef-a3e8-487d-87bc-f0a868b2a484 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:20.502652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:20.503712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.505316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 483/962] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:20.506192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199181ef-a3e8-487d-87bc-f0a868b2a484 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.507533 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c78ed-842c-43b6-8f39-9c194445d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:20.508151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c78ed-842c-43b6-8f39-9c194445d31f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.508436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c78ed-842c-43b6-8f39-9c194445d31f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.508699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c78ed-842c-43b6-8f39-9c194445d31f None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.509357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c78ed-842c-43b6-8f39-9c194445d31f None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:20.509658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c78ed-842c-43b6-8f39-9c194445d31f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:20.511637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c78ed-842c-43b6-8f39-9c194445d31f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:20.529895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 480/963] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:15:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:20.530721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c78ed-842c-43b6-8f39-9c194445d31f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.579070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:20.625818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199181ef-a3e8-487d-87bc-f0a868b2a484 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.648366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c78ed-842c-43b6-8f39-9c194445d31f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.662655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78400a9-6e34-4fa9-b046-e3cf8b480a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:20.663292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78400a9-6e34-4fa9-b046-e3cf8b480a80 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.663574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78400a9-6e34-4fa9-b046-e3cf8b480a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.663847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78400a9-6e34-4fa9-b046-e3cf8b480a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.674588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78400a9-6e34-4fa9-b046-e3cf8b480a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:20.676064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78400a9-6e34-4fa9-b046-e3cf8b480a80 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:20.689145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c9a0-e29b-4190-a638-4b6d988573aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:20.690078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c9a0-e29b-4190-a638-4b6d988573aa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.690529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c9a0-e29b-4190-a638-4b6d988573aa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.690959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c9a0-e29b-4190-a638-4b6d988573aa None admin] PATH_INFO: `/v3/services/compute` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.691738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c9a0-e29b-4190-a638-4b6d988573aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-e78400a9-6e34-4fa9-b046-e3cf8b480a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:20.692503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:20.693629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:20.694573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:20.694573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:20.694573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:20.694573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 484/964] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:20.695452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78400a9-6e34-4fa9-b046-e3cf8b480a80 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.696361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c9a0-e29b-4190-a638-4b6d988573aa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-8665c9a0-e29b-4190-a638-4b6d988573aa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:20.711851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:20.713005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 05:15:20.714680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:20.715987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 481/965] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:20.717439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c9a0-e29b-4190-a638-4b6d988573aa None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.763860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:20.820225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78400a9-6e34-4fa9-b046-e3cf8b480a80 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.843224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c9a0-e29b-4190-a638-4b6d988573aa None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.845783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:20.857879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0ef78-4efe-4911-9022-56c70fcc68bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:20.858747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0ef78-4efe-4911-9022-56c70fcc68bb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.858892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0ef78-4efe-4911-9022-56c70fcc68bb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.859209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0ef78-4efe-4911-9022-56c70fcc68bb None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.859687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0ef78-4efe-4911-9022-56c70fcc68bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:20.859988 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0ef78-4efe-4911-9022-56c70fcc68bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:20.861361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0ef78-4efe-4911-9022-56c70fcc68bb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:20.874162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 485/966] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:20.874162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0ef78-4efe-4911-9022-56c70fcc68bb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.884160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be51ccd-e036-4873-9dc9-1918354399f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:20.885086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be51ccd-e036-4873-9dc9-1918354399f3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:20.885518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be51ccd-e036-4873-9dc9-1918354399f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:20.886415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be51ccd-e036-4873-9dc9-1918354399f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:20.887055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be51ccd-e036-4873-9dc9-1918354399f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:20.887356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be51ccd-e036-4873-9dc9-1918354399f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:20.888731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be51ccd-e036-4873-9dc9-1918354399f3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:20.901081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 482/967] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:20.901956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be51ccd-e036-4873-9dc9-1918354399f3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:20.981114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0ef78-4efe-4911-9022-56c70fcc68bb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:20.994941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:21.007826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be51ccd-e036-4873-9dc9-1918354399f3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:21.018823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd6a3f-d769-4f89-98d7-7c2116b157c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:21.019079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd6a3f-d769-4f89-98d7-7c2116b157c3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:21.019359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd6a3f-d769-4f89-98d7-7c2116b157c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:21.019632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd6a3f-d769-4f89-98d7-7c2116b157c3 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:21.020257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd6a3f-d769-4f89-98d7-7c2116b157c3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:21.020562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd6a3f-d769-4f89-98d7-7c2116b157c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:21.021923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd6a3f-d769-4f89-98d7-7c2116b157c3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:21.031129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 486/968] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:15:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:21.031129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd6a3f-d769-4f89-98d7-7c2116b157c3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:21.051561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bccee-a4f2-4504-8dc2-59f21772cacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:21.052167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bccee-a4f2-4504-8dc2-59f21772cacd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:21.052693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bccee-a4f2-4504-8dc2-59f21772cacd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:21.052693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bccee-a4f2-4504-8dc2-59f21772cacd None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:21.053325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bccee-a4f2-4504-8dc2-59f21772cacd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:21.053631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bccee-a4f2-4504-8dc2-59f21772cacd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:21.055017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bccee-a4f2-4504-8dc2-59f21772cacd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:21.060367 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 483/969] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:21.061140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bccee-a4f2-4504-8dc2-59f21772cacd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:21.092083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:21.141635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd6a3f-d769-4f89-98d7-7c2116b157c3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:21.176533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90341db4-68a7-42db-9542-fa70d015646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:21.177160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90341db4-68a7-42db-9542-fa70d015646d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:21.177441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90341db4-68a7-42db-9542-fa70d015646d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:21.177714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90341db4-68a7-42db-9542-fa70d015646d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:21.177812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bccee-a4f2-4504-8dc2-59f21772cacd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:21.185037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90341db4-68a7-42db-9542-fa70d015646d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:21.186515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90341db4-68a7-42db-9542-fa70d015646d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-90341db4-68a7-42db-9542-fa70d015646d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:21.197818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:21.198742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:21.199657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:21.199657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:21.199657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:21.199657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:21.199657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:21.199657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:21.199657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:21.199657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:21.199657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 487/970] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:21.200582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90341db4-68a7-42db-9542-fa70d015646d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:21.220630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcabb9-311a-48b5-b123-04e9c90f51b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:21.221222 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcabb9-311a-48b5-b123-04e9c90f51b8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:21.221499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcabb9-311a-48b5-b123-04e9c90f51b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:21.221773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcabb9-311a-48b5-b123-04e9c90f51b8 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:21.222415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcabb9-311a-48b5-b123-04e9c90f51b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:21.222716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcabb9-311a-48b5-b123-04e9c90f51b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:21.224057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcabb9-311a-48b5-b123-04e9c90f51b8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:21.229311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 484/971] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:15:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:21.235105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcabb9-311a-48b5-b123-04e9c90f51b8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:21.240506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:21.317640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90341db4-68a7-42db-9542-fa70d015646d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:21.356477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847d90-8cc8-4ccc-8283-f7759ead325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:21.356725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847d90-8cc8-4ccc-8283-f7759ead325e None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:21.356969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847d90-8cc8-4ccc-8283-f7759ead325e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:21.357220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847d90-8cc8-4ccc-8283-f7759ead325e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:21.357978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c847d90-8cc8-4ccc-8283-f7759ead325e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:21.359433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c847d90-8cc8-4ccc-8283-f7759ead325e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:21.361767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:21.375007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcabb9-311a-48b5-b123-04e9c90f51b8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:21.386069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 488/972] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:15:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:21.426025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69709e20-32f7-4ac6-a238-18dec27c920c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:21.427053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69709e20-32f7-4ac6-a238-18dec27c920c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:21.427572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69709e20-32f7-4ac6-a238-18dec27c920c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:21.428048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69709e20-32f7-4ac6-a238-18dec27c920c None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:21.428979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69709e20-32f7-4ac6-a238-18dec27c920c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:21.429503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69709e20-32f7-4ac6-a238-18dec27c920c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:21.431163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69709e20-32f7-4ac6-a238-18dec27c920c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:21.445669 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 485/973] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:21.452139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847d90-8cc8-4ccc-8283-f7759ead325e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:21.562627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:21.573557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847d90-8cc8-4ccc-8283-f7759ead325e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:21.615931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62713be-a33a-44c6-87ee-fec874f95a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:21.616549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62713be-a33a-44c6-87ee-fec874f95a76 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:21.616828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62713be-a33a-44c6-87ee-fec874f95a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:21.617102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62713be-a33a-44c6-87ee-fec874f95a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:21.617833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62713be-a33a-44c6-87ee-fec874f95a76 None admin] RBAC: Adding query filter params (role.id=688834a9cd994fa08909c0296f33b5a0, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d2297673f49b4ce8bf40ac5f750d989f) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:21.618178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62713be-a33a-44c6-87ee-fec874f95a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:21.619620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62713be-a33a-44c6-87ee-fec874f95a76 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:21.626475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 489/974] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:21 2022] GET /identity/v3/role_assignments?user.id=d2297673f49b4ce8bf40ac5f750d989f&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=688834a9cd994fa08909c0296f33b5a0 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:22.790218 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee19a57-738b-4fd8-af39-828f28586cda None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:22.790752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee19a57-738b-4fd8-af39-828f28586cda None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:22.790752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee19a57-738b-4fd8-af39-828f28586cda None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:22.792862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 486/975] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:22.798342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d7481-ec94-4308-9475-00566924f77a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:22.798743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d7481-ec94-4308-9475-00566924f77a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:22.798917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d7481-ec94-4308-9475-00566924f77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:22.818118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-623d7481-ec94-4308-9475-00566924f77a None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:22.819292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-623d7481-ec94-4308-9475-00566924f77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:22.887880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 490/976] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:23.157755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee19a57-738b-4fd8-af39-828f28586cda None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:23.267751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee19a57-738b-4fd8-af39-828f28586cda None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:23.303633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bedda-308b-422b-9d97-f7624b4ba276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:23.304240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bedda-308b-422b-9d97-f7624b4ba276 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:23.304517 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bedda-308b-422b-9d97-f7624b4ba276 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:23.304788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bedda-308b-422b-9d97-f7624b4ba276 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:23.305498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bedda-308b-422b-9d97-f7624b4ba276 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:23.306934 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bedda-308b-422b-9d97-f7624b4ba276 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:23.309493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WARNING keystone.server.flask.common [None req-ac1bedda-308b-422b-9d97-f7624b4ba276 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 21 05:15:23.324011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.sql.core [None req-ac1bedda-308b-422b-9d97-f7624b4ba276 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 21 05:15:23.324011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 21 05:15:23.324011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [parameters: {'user_id': '0841bba38670402587d8989d7d9edec0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 21 05:15:23.324011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81410) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ac1bedda-308b-422b-9d97-f7624b4ba276 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application conn.query(q) Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result.read() Sep 21 05:15:23.342037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 05:15:23.342967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application next(self.gen) Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application yield resource Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application next(self.gen) Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.session.rollback() Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.force_reraise() Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise self.value Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application session.commit() Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 21 05:15:23.343876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._prepare_impl() Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.session.flush() Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._flush(objects) Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application compat.raise_( Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application flush_context.execute() Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rec.execute(self) Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 21 05:15:23.345107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 05:15:23.345974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application conn.query(q) Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result.read() Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 05:15:23.346790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [parameters: {'user_id': '0841bba38670402587d8989d7d9edec0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:23.348524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 21 05:15:23.350415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 21 05:15:23.352729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 21 05:15:23.352729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 21 05:15:23.352729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 21 05:15:23.352729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 21 05:15:23.352729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 21 05:15:23.352729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:23.352729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 487/977] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Sep 21 05:15:23 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:23.354677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d7481-ec94-4308-9475-00566924f77a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:23.446075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:23.462225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d7481-ec94-4308-9475-00566924f77a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:23.499352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8ec5b-b6a1-422d-bcab-50bfb72c05ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:23.500040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8ec5b-b6a1-422d-bcab-50bfb72c05ad None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:23.500321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8ec5b-b6a1-422d-bcab-50bfb72c05ad None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:23.500593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8ec5b-b6a1-422d-bcab-50bfb72c05ad None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:23.507968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8ec5b-b6a1-422d-bcab-50bfb72c05ad None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:23.509472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8ec5b-b6a1-422d-bcab-50bfb72c05ad None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-64f8ec5b-b6a1-422d-bcab-50bfb72c05ad None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:23.521130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:23.523303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 05:15:23.524592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:23.526435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 491/978] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:15:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:23.528989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8123-cc82-49d7-903d-87213ce3a3a7 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:23.529276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8123-cc82-49d7-903d-87213ce3a3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:23.529550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8123-cc82-49d7-903d-87213ce3a3a7 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:23.531219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8ec5b-b6a1-422d-bcab-50bfb72c05ad None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:23.537382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 488/979] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:23.542526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b54fe-bd9d-4e4f-b391-421d1b7f1b01 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:23.542840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b54fe-bd9d-4e4f-b391-421d1b7f1b01 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:23.543074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b54fe-bd9d-4e4f-b391-421d1b7f1b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:23.569395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-302b54fe-bd9d-4e4f-b391-421d1b7f1b01 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:23.571334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-302b54fe-bd9d-4e4f-b391-421d1b7f1b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:23.576360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:23.663744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 489/980] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:23.665089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2e97b-c286-46fa-a907-e3c92c074a23 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:23.665378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2e97b-c286-46fa-a907-e3c92c074a23 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:23.665652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2e97b-c286-46fa-a907-e3c92c074a23 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:23.667689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 490/981] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:23.672348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0cf42-6780-4788-9766-80a067374bba None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:23.672629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0cf42-6780-4788-9766-80a067374bba None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:23.672903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0cf42-6780-4788-9766-80a067374bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:23.679123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8ec5b-b6a1-422d-bcab-50bfb72c05ad None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:23.693099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-aad0cf42-6780-4788-9766-80a067374bba None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:23.694181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aad0cf42-6780-4788-9766-80a067374bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:23.726091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430d31e-c8df-42e3-a196-bb81bd4cf9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:23.727247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430d31e-c8df-42e3-a196-bb81bd4cf9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:23.727697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430d31e-c8df-42e3-a196-bb81bd4cf9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:23.728380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430d31e-c8df-42e3-a196-bb81bd4cf9b4 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:23.730253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430d31e-c8df-42e3-a196-bb81bd4cf9b4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:23.730506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430d31e-c8df-42e3-a196-bb81bd4cf9b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:23.731985 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430d31e-c8df-42e3-a196-bb81bd4cf9b4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:23.774541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 492/982] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Sep 21 05:15:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:23.783663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 491/983] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:23.841787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16d130-dbc5-43a2-b030-658d2176becf None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:23.842365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16d130-dbc5-43a2-b030-658d2176becf None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:23.842365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16d130-dbc5-43a2-b030-658d2176becf None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:23.843988 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 493/984] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:23.849537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a984f-448c-46ff-8344-b0c24abfe4c4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:23.850428 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a984f-448c-46ff-8344-b0c24abfe4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:23.850855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a984f-448c-46ff-8344-b0c24abfe4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:23.871983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fa2a984f-448c-46ff-8344-b0c24abfe4c4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:23.873438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fa2a984f-448c-46ff-8344-b0c24abfe4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:23.945265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:23.951149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 492/985] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:23.962465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16d130-dbc5-43a2-b030-658d2176becf None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:23.966862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:24.057097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a984f-448c-46ff-8344-b0c24abfe4c4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:24.077283 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16d130-dbc5-43a2-b030-658d2176becf None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:24.114816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498190c9-2616-4953-a7ae-311afd9e364e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:24.115417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498190c9-2616-4953-a7ae-311afd9e364e None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:24.115695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498190c9-2616-4953-a7ae-311afd9e364e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:24.115973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498190c9-2616-4953-a7ae-311afd9e364e None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:24.116625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498190c9-2616-4953-a7ae-311afd9e364e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:24.118079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498190c9-2616-4953-a7ae-311afd9e364e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:24.133167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 494/986] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:15:23 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:24.175346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a984f-448c-46ff-8344-b0c24abfe4c4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:24.223781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4473a-f7ac-4ef6-bf21-aec8497ef2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:24.224427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4473a-f7ac-4ef6-bf21-aec8497ef2fb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:24.224701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4473a-f7ac-4ef6-bf21-aec8497ef2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:24.224967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4473a-f7ac-4ef6-bf21-aec8497ef2fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:24.225549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4473a-f7ac-4ef6-bf21-aec8497ef2fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:24.227122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4473a-f7ac-4ef6-bf21-aec8497ef2fb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:24.229838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498190c9-2616-4953-a7ae-311afd9e364e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d7a4473a-f7ac-4ef6-bf21-aec8497ef2fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:24.236884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:24.238579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:24.239500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:24.243462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 493/987] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:24.244206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4473a-f7ac-4ef6-bf21-aec8497ef2fb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:24.301904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:24.309999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:24.355378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498190c9-2616-4953-a7ae-311afd9e364e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:24.360464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4473a-f7ac-4ef6-bf21-aec8497ef2fb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:24.389735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d010d9-ea83-4643-a42f-1241db8198cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:24.390393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d010d9-ea83-4643-a42f-1241db8198cc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:24.390674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d010d9-ea83-4643-a42f-1241db8198cc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:24.390946 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d010d9-ea83-4643-a42f-1241db8198cc None admin] PATH_INFO: `/v3/services/image` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:24.391512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d010d9-ea83-4643-a42f-1241db8198cc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:24.392874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d010d9-ea83-4643-a42f-1241db8198cc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:24.397854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060fff5-87e9-49a7-9e24-bdbf8773619a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:24.398477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060fff5-87e9-49a7-9e24-bdbf8773619a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:24.398760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060fff5-87e9-49a7-9e24-bdbf8773619a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:24.399030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060fff5-87e9-49a7-9e24-bdbf8773619a None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:24.399642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060fff5-87e9-49a7-9e24-bdbf8773619a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:24.399940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060fff5-87e9-49a7-9e24-bdbf8773619a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:24.401840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060fff5-87e9-49a7-9e24-bdbf8773619a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-66d010d9-ea83-4643-a42f-1241db8198cc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:24.402075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:24.403033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:24.407018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 495/988] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Sep 21 05:15:24 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:24.409142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d010d9-ea83-4643-a42f-1241db8198cc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:24.412963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 494/989] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:24.419021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060fff5-87e9-49a7-9e24-bdbf8773619a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:24.517566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d010d9-ea83-4643-a42f-1241db8198cc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:24.524683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060fff5-87e9-49a7-9e24-bdbf8773619a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:24.548873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:24.550120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:24.560816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f7c90-ded8-41e9-bbab-24b0f9c9cd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:24.561035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83734f92-2cb8-42d2-9179-ccc94f716280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:24.561487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7c90-ded8-41e9-bbab-24b0f9c9cd26 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:24.561642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83734f92-2cb8-42d2-9179-ccc94f716280 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:24.561881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7c90-ded8-41e9-bbab-24b0f9c9cd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:24.562005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83734f92-2cb8-42d2-9179-ccc94f716280 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:24.562259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83734f92-2cb8-42d2-9179-ccc94f716280 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:24.562373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7c90-ded8-41e9-bbab-24b0f9c9cd26 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:24.562882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83734f92-2cb8-42d2-9179-ccc94f716280 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:24.563050 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f7c90-ded8-41e9-bbab-24b0f9c9cd26 None admin] RBAC: Adding query filter params (name=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:24.563186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83734f92-2cb8-42d2-9179-ccc94f716280 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:24.563460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f7c90-ded8-41e9-bbab-24b0f9c9cd26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:24.564532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83734f92-2cb8-42d2-9179-ccc94f716280 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:24.564918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f7c90-ded8-41e9-bbab-24b0f9c9cd26 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:24.569687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 495/990] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:24.576007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 496/991] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:24.576160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83734f92-2cb8-42d2-9179-ccc94f716280 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:24.576896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f7c90-ded8-41e9-bbab-24b0f9c9cd26 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:24.682308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83734f92-2cb8-42d2-9179-ccc94f716280 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:24.692774 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f7c90-ded8-41e9-bbab-24b0f9c9cd26 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:24.718604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c5738-e14c-4fc8-89b0-f5a5e2f38cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:24.719357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c5738-e14c-4fc8-89b0-f5a5e2f38cff None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:24.719654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c5738-e14c-4fc8-89b0-f5a5e2f38cff None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:24.720113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c5738-e14c-4fc8-89b0-f5a5e2f38cff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:24.721369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c5738-e14c-4fc8-89b0-f5a5e2f38cff None admin] RBAC: Adding query filter params (interface=admin, service_id=5f8fb93ee1074f9c88e2a7d2b87bf591, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:24.721749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c5738-e14c-4fc8-89b0-f5a5e2f38cff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:24.723151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c5738-e14c-4fc8-89b0-f5a5e2f38cff None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:24.728817 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 496/992] 10.222.0.18 () {56 vars in 1249 bytes} [Wed Sep 21 05:15:24 2022] GET /identity/v3/endpoints?service_id=5f8fb93ee1074f9c88e2a7d2b87bf591&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:24.733981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c5738-e14c-4fc8-89b0-f5a5e2f38cff None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:24.735139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74eb5b-a470-46bd-8c1e-a80b1789897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:24.735756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74eb5b-a470-46bd-8c1e-a80b1789897a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:24.736037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74eb5b-a470-46bd-8c1e-a80b1789897a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:24.736444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74eb5b-a470-46bd-8c1e-a80b1789897a None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:24.737095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74eb5b-a470-46bd-8c1e-a80b1789897a None admin] RBAC: Adding query filter params (type=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:24.737748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74eb5b-a470-46bd-8c1e-a80b1789897a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:24.739155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74eb5b-a470-46bd-8c1e-a80b1789897a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:24.753987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 497/993] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:24 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:24.801685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:24.854062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c5738-e14c-4fc8-89b0-f5a5e2f38cff None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:24.891698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6344a-d025-406e-a269-8ce6e8ee8420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:24.892326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6344a-d025-406e-a269-8ce6e8ee8420 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:24.892604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6344a-d025-406e-a269-8ce6e8ee8420 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:24.892876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6344a-d025-406e-a269-8ce6e8ee8420 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:24.893560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6344a-d025-406e-a269-8ce6e8ee8420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:24.894975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6344a-d025-406e-a269-8ce6e8ee8420 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:24.901997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 497/994] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Sep 21 05:15:24 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:15:26.310955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7cf1a-d0ab-4099-a66f-deb1f46c6bfa None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:26.311497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7cf1a-d0ab-4099-a66f-deb1f46c6bfa None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:26.311497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7cf1a-d0ab-4099-a66f-deb1f46c6bfa None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:26.312845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 498/995] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:26.317849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b7d6b-778a-4c0b-a0c5-35a5b94d964b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:26.318251 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b7d6b-778a-4c0b-a0c5-35a5b94d964b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:26.318449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b7d6b-778a-4c0b-a0c5-35a5b94d964b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:26.336827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-cd9b7d6b-778a-4c0b-a0c5-35a5b94d964b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:26.338023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cd9b7d6b-778a-4c0b-a0c5-35a5b94d964b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:26.403955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 498/996] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:26.666556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7cf1a-d0ab-4099-a66f-deb1f46c6bfa None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:26.724398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:26.771508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7cf1a-d0ab-4099-a66f-deb1f46c6bfa None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:26.808963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfa91d-2db7-439c-9429-d1287c529005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:26.809614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfa91d-2db7-439c-9429-d1287c529005 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:26.809889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfa91d-2db7-439c-9429-d1287c529005 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:26.810187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfa91d-2db7-439c-9429-d1287c529005 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:26.810776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfa91d-2db7-439c-9429-d1287c529005 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:26.812288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfa91d-2db7-439c-9429-d1287c529005 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5ebfa91d-2db7-439c-9429-d1287c529005 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:26.821628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:26.822666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 05:15:26.823499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:26.826270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 499/997] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:26.828592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b7d6b-778a-4c0b-a0c5-35a5b94d964b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:26.874899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:26.907340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c74e7a-1114-4401-afe8-ccd1b0450510 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:26.907626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c74e7a-1114-4401-afe8-ccd1b0450510 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:26.907906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c74e7a-1114-4401-afe8-ccd1b0450510 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:26.909232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 500/998] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:26.914116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be073130-ac4a-4961-a238-d547a934ba1e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:26.914395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be073130-ac4a-4961-a238-d547a934ba1e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:26.914672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be073130-ac4a-4961-a238-d547a934ba1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:26.930816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b7d6b-778a-4c0b-a0c5-35a5b94d964b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:26.933303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-be073130-ac4a-4961-a238-d547a934ba1e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:26.933882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-be073130-ac4a-4961-a238-d547a934ba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:26.969192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f1b84-7c5d-4509-82de-4c54ced45f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:26.970152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f1b84-7c5d-4509-82de-4c54ced45f60 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:26.970589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f1b84-7c5d-4509-82de-4c54ced45f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:26.971016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f1b84-7c5d-4509-82de-4c54ced45f60 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:26.971653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f1b84-7c5d-4509-82de-4c54ced45f60 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:26.971955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f1b84-7c5d-4509-82de-4c54ced45f60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:26.973352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f1b84-7c5d-4509-82de-4c54ced45f60 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:26.983961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 499/999] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:15:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:26.985507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f1b84-7c5d-4509-82de-4c54ced45f60 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:27.005027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 501/1000] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:27.075641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a9c12-a91e-412b-b641-1b4953940b75 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:27.075974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a9c12-a91e-412b-b641-1b4953940b75 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:27.076197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a9c12-a91e-412b-b641-1b4953940b75 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:27.077529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 502/1001] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:27.082633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634dc72-df52-4320-bea5-a0acd0f9e722 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:27.082916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634dc72-df52-4320-bea5-a0acd0f9e722 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:27.083193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634dc72-df52-4320-bea5-a0acd0f9e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:27.090098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f1b84-7c5d-4509-82de-4c54ced45f60 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:27.101615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2634dc72-df52-4320-bea5-a0acd0f9e722 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:27.102590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:27.102833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2634dc72-df52-4320-bea5-a0acd0f9e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:27.128436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a5196-1043-4afc-8f13-eab656d055ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:27.129005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a5196-1043-4afc-8f13-eab656d055ce None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:27.129277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a5196-1043-4afc-8f13-eab656d055ce None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:27.129549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a5196-1043-4afc-8f13-eab656d055ce None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:27.130188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a5196-1043-4afc-8f13-eab656d055ce None admin] RBAC: Adding query filter params (type=compute) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:27.130489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a5196-1043-4afc-8f13-eab656d055ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:27.131824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a5196-1043-4afc-8f13-eab656d055ce None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:27.137330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 500/1002] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:15:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:27.142661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a5196-1043-4afc-8f13-eab656d055ce None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:27.158015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:27.180997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 503/1003] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:27.245225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a5196-1043-4afc-8f13-eab656d055ce None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:27.269764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634dc72-df52-4320-bea5-a0acd0f9e722 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:27.279724 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a9297-3b08-432a-b9b8-83a019c20d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:27.280341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a9297-3b08-432a-b9b8-83a019c20d3f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:27.280614 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a9297-3b08-432a-b9b8-83a019c20d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:27.280888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a9297-3b08-432a-b9b8-83a019c20d3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:27.281579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a9297-3b08-432a-b9b8-83a019c20d3f None admin] RBAC: Adding query filter params (interface=public, service_id=427acd936eef4f31a9eeb6eacf5fa0b8, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:27.281885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a9297-3b08-432a-b9b8-83a019c20d3f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:27.283268 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a9297-3b08-432a-b9b8-83a019c20d3f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:27.292867 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 501/1004] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Sep 21 05:15:27 2022] GET /identity/v3/endpoints?service_id=427acd936eef4f31a9eeb6eacf5fa0b8&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:27.293787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a9297-3b08-432a-b9b8-83a019c20d3f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:27.339288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:27.384153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634dc72-df52-4320-bea5-a0acd0f9e722 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:27.400763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a9297-3b08-432a-b9b8-83a019c20d3f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:27.427729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0f81a-79bd-44d2-8d65-2749e95a43ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:27.428443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0f81a-79bd-44d2-8d65-2749e95a43ac None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:27.428709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0f81a-79bd-44d2-8d65-2749e95a43ac None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:27.428977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0f81a-79bd-44d2-8d65-2749e95a43ac None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:27.429633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0f81a-79bd-44d2-8d65-2749e95a43ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:27.431038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0f81a-79bd-44d2-8d65-2749e95a43ac None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:27.440497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ff895-97a5-4317-979b-7ebcd6c28bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:27.441131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ff895-97a5-4317-979b-7ebcd6c28bfb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:27.441411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ff895-97a5-4317-979b-7ebcd6c28bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:27.441685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ff895-97a5-4317-979b-7ebcd6c28bfb None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:27.442340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ff895-97a5-4317-979b-7ebcd6c28bfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:27.443683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ff895-97a5-4317-979b-7ebcd6c28bfb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:27.449452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 504/1005] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:15:27 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:15:27.453976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 502/1006] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Sep 21 05:15:27 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:15:27.462548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0f81a-79bd-44d2-8d65-2749e95a43ac None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:27.513554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:27.594587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0f81a-79bd-44d2-8d65-2749e95a43ac None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:27.633796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf5d9f-1d6a-4422-b9f7-a371efa33401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:27.634540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf5d9f-1d6a-4422-b9f7-a371efa33401 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:27.634767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf5d9f-1d6a-4422-b9f7-a371efa33401 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:27.635040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf5d9f-1d6a-4422-b9f7-a371efa33401 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:27.635676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf5d9f-1d6a-4422-b9f7-a371efa33401 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:27.637077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf5d9f-1d6a-4422-b9f7-a371efa33401 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0faf5d9f-1d6a-4422-b9f7-a371efa33401 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:27.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:27.648421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:27.649391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 505/1007] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Sep 21 05:15:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:27.655058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ff895-97a5-4317-979b-7ebcd6c28bfb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:27.765406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ff895-97a5-4317-979b-7ebcd6c28bfb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:27.766985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:27.801582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99c51a-0601-4f29-8950-5295e2aa925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:27.802222 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99c51a-0601-4f29-8950-5295e2aa925b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:27.802503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99c51a-0601-4f29-8950-5295e2aa925b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:27.802776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99c51a-0601-4f29-8950-5295e2aa925b None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:27.803394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99c51a-0601-4f29-8950-5295e2aa925b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:27.803698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99c51a-0601-4f29-8950-5295e2aa925b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:27.805047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99c51a-0601-4f29-8950-5295e2aa925b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:27.810191 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 503/1008] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:27.815847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf5d9f-1d6a-4422-b9f7-a371efa33401 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:27.913882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:27.917894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf5d9f-1d6a-4422-b9f7-a371efa33401 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:27.952164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1d68f-0732-4fa3-9462-f196bf3f9f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:27.952783 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d68f-0732-4fa3-9462-f196bf3f9f0d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:27.953055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d68f-0732-4fa3-9462-f196bf3f9f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:27.953325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d68f-0732-4fa3-9462-f196bf3f9f0d None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:27.953969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d68f-0732-4fa3-9462-f196bf3f9f0d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:27.954274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d68f-0732-4fa3-9462-f196bf3f9f0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:27.956457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d68f-0732-4fa3-9462-f196bf3f9f0d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:27.960852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 506/1009] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Sep 21 05:15:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:27.967285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99c51a-0601-4f29-8950-5295e2aa925b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:28.067774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99c51a-0601-4f29-8950-5295e2aa925b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:28.101764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cda78d-03d7-4dc4-b1b7-feafd80996c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:28.102401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cda78d-03d7-4dc4-b1b7-feafd80996c9 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:28.102684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cda78d-03d7-4dc4-b1b7-feafd80996c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:28.102956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cda78d-03d7-4dc4-b1b7-feafd80996c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:28.103624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cda78d-03d7-4dc4-b1b7-feafd80996c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:28.104996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cda78d-03d7-4dc4-b1b7-feafd80996c9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:28.122693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 504/1010] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:15:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:29.626904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a25181-d49e-498f-add0-5828a9767be3 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:29.627447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a25181-d49e-498f-add0-5828a9767be3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:29.627447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a25181-d49e-498f-add0-5828a9767be3 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:29.628938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 507/1011] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:29.633841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154f87-50ab-4c99-85d2-596ca02f1843 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:29.634241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154f87-50ab-4c99-85d2-596ca02f1843 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:29.634416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154f87-50ab-4c99-85d2-596ca02f1843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:29.652524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d8154f87-50ab-4c99-85d2-596ca02f1843 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:29.653704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d8154f87-50ab-4c99-85d2-596ca02f1843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:29.672754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3ccb5-e42d-4e52-b0e7-41792044d1b5 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:29.673043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3ccb5-e42d-4e52-b0e7-41792044d1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:29.673318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3ccb5-e42d-4e52-b0e7-41792044d1b5 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:29.674936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 508/1012] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:29.675265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:29.680080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2547e-b0f1-461d-b155-b698858703c0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:29.680370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2547e-b0f1-461d-b155-b698858703c0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:29.680643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2547e-b0f1-461d-b155-b698858703c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:29.700879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-97f2547e-b0f1-461d-b155-b698858703c0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:29.702111 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-97f2547e-b0f1-461d-b155-b698858703c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:29.735100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 505/1013] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:29 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:29.775260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 509/1014] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:29 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:30.005786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8154f87-50ab-4c99-85d2-596ca02f1843 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.049723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2547e-b0f1-461d-b155-b698858703c0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.113218 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8154f87-50ab-4c99-85d2-596ca02f1843 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.150914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:30.156907 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda839ef-0e0d-4638-a4a1-f89206480cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.157798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda839ef-0e0d-4638-a4a1-f89206480cbe None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.158136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda839ef-0e0d-4638-a4a1-f89206480cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.158410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda839ef-0e0d-4638-a4a1-f89206480cbe None admin] PATH_INFO: `/v3/services/compute` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.159003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda839ef-0e0d-4638-a4a1-f89206480cbe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.159235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2547e-b0f1-461d-b155-b698858703c0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.160335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda839ef-0e0d-4638-a4a1-f89206480cbe None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-fda839ef-0e0d-4638-a4a1-f89206480cbe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:30.172265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:30.173403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:30.175202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 506/1015] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:30.179674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda839ef-0e0d-4638-a4a1-f89206480cbe None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.198920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f00fe2-5b42-4e97-889b-01483ba78c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.199573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f00fe2-5b42-4e97-889b-01483ba78c7b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.199851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f00fe2-5b42-4e97-889b-01483ba78c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.200123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f00fe2-5b42-4e97-889b-01483ba78c7b None admin] PATH_INFO: `/v3/services/image` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.200717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f00fe2-5b42-4e97-889b-01483ba78c7b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.202155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f00fe2-5b42-4e97-889b-01483ba78c7b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.204623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-f9f00fe2-5b42-4e97-889b-01483ba78c7b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:30.214325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:30.215443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:30.216295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 510/1016] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Sep 21 05:15:30 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:30.221248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f00fe2-5b42-4e97-889b-01483ba78c7b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.302525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda839ef-0e0d-4638-a4a1-f89206480cbe None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.324630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f00fe2-5b42-4e97-889b-01483ba78c7b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.336362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da7d41-9a13-43f8-ba7b-9b86cb517115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.336964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da7d41-9a13-43f8-ba7b-9b86cb517115 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.337242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da7d41-9a13-43f8-ba7b-9b86cb517115 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.337515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da7d41-9a13-43f8-ba7b-9b86cb517115 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.338157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da7d41-9a13-43f8-ba7b-9b86cb517115 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:30.338520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da7d41-9a13-43f8-ba7b-9b86cb517115 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.340379 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da7d41-9a13-43f8-ba7b-9b86cb517115 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.345050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 507/1017] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:15:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:30.346198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f51dd-f62c-4e41-acca-30e51881d524 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.346488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f51dd-f62c-4e41-acca-30e51881d524 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.346752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f51dd-f62c-4e41-acca-30e51881d524 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.348163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 508/1018] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:30.350451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f51dd-f62c-4e41-acca-30e51881d524 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.361278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee0fa1-129d-4b74-be09-c442506eefaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.361889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee0fa1-129d-4b74-be09-c442506eefaa None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.362201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee0fa1-129d-4b74-be09-c442506eefaa None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.362736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee0fa1-129d-4b74-be09-c442506eefaa None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.363363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee0fa1-129d-4b74-be09-c442506eefaa None admin] RBAC: Adding query filter params (name=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:30.363665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee0fa1-129d-4b74-be09-c442506eefaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.365032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee0fa1-129d-4b74-be09-c442506eefaa None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.369801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 511/1019] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:30.370847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69a5e1-f219-4ca9-9b99-3997be5d4239 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.371128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69a5e1-f219-4ca9-9b99-3997be5d4239 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.371398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69a5e1-f219-4ca9-9b99-3997be5d4239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.389193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1f69a5e1-f219-4ca9-9b99-3997be5d4239 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:30.390408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1f69a5e1-f219-4ca9-9b99-3997be5d4239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:30.407705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:30.439429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:30.457635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 512/1020] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:30 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:30.458382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69a5e1-f219-4ca9-9b99-3997be5d4239 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.460088 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f51dd-f62c-4e41-acca-30e51881d524 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.498223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f4a47-6686-467c-8f5d-d2cb3bc6f668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.498834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f4a47-6686-467c-8f5d-d2cb3bc6f668 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.499121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f4a47-6686-467c-8f5d-d2cb3bc6f668 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.499398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f4a47-6686-467c-8f5d-d2cb3bc6f668 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.500020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f4a47-6686-467c-8f5d-d2cb3bc6f668 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:30.500323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f4a47-6686-467c-8f5d-d2cb3bc6f668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.501670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f4a47-6686-467c-8f5d-d2cb3bc6f668 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.506384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 509/1021] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Sep 21 05:15:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:30.512433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f4a47-6686-467c-8f5d-d2cb3bc6f668 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.561882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69a5e1-f219-4ca9-9b99-3997be5d4239 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.599205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba785098-7cdd-4aae-94f0-6a84aacb471b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.599819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba785098-7cdd-4aae-94f0-6a84aacb471b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.600108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba785098-7cdd-4aae-94f0-6a84aacb471b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.600388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba785098-7cdd-4aae-94f0-6a84aacb471b None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.601015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba785098-7cdd-4aae-94f0-6a84aacb471b None admin] RBAC: Adding query filter params (type=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:30.601321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba785098-7cdd-4aae-94f0-6a84aacb471b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.602716 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba785098-7cdd-4aae-94f0-6a84aacb471b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.607627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 513/1022] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:30.613617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba785098-7cdd-4aae-94f0-6a84aacb471b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.630629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f4a47-6686-467c-8f5d-d2cb3bc6f668 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.631658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:30.667146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664d26c-1c26-47b8-8295-73f0c364dde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.667757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664d26c-1c26-47b8-8295-73f0c364dde4 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.668025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664d26c-1c26-47b8-8295-73f0c364dde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.668297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664d26c-1c26-47b8-8295-73f0c364dde4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.668954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664d26c-1c26-47b8-8295-73f0c364dde4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.670404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664d26c-1c26-47b8-8295-73f0c364dde4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.687928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:30.695037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 510/1023] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:15:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:15:30.721076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba785098-7cdd-4aae-94f0-6a84aacb471b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.731768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664d26c-1c26-47b8-8295-73f0c364dde4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.770680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d661a-6ce5-4de3-a0ce-bbda50fd3eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.771311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d661a-6ce5-4de3-a0ce-bbda50fd3eee None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.771595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d661a-6ce5-4de3-a0ce-bbda50fd3eee None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.771869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d661a-6ce5-4de3-a0ce-bbda50fd3eee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.772575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d661a-6ce5-4de3-a0ce-bbda50fd3eee None admin] RBAC: Adding query filter params (interface=public, service_id=8ca5f45936334aafbeedd9efac1c502b, region_id=RegionOne) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:30.772886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d661a-6ce5-4de3-a0ce-bbda50fd3eee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.774318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d661a-6ce5-4de3-a0ce-bbda50fd3eee None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.780153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 514/1024] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Sep 21 05:15:30 2022] GET /identity/v3/endpoints?service_id=8ca5f45936334aafbeedd9efac1c502b&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:30.785905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d661a-6ce5-4de3-a0ce-bbda50fd3eee None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.852197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664d26c-1c26-47b8-8295-73f0c364dde4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.872069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:30.894126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587881bc-dead-4c09-9617-e05d420d9dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.894477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587881bc-dead-4c09-9617-e05d420d9dfa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.894748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587881bc-dead-4c09-9617-e05d420d9dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.895020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587881bc-dead-4c09-9617-e05d420d9dfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.897757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587881bc-dead-4c09-9617-e05d420d9dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.899050 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d661a-6ce5-4de3-a0ce-bbda50fd3eee None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:30.899423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587881bc-dead-4c09-9617-e05d420d9dfa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.901183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 511/1025] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:30.906800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587881bc-dead-4c09-9617-e05d420d9dfa None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:30.940331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d7be4-3ffa-460e-9c4c-579273d98ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:30.940941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d7be4-3ffa-460e-9c4c-579273d98ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:30.941250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d7be4-3ffa-460e-9c4c-579273d98ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:30.941483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d7be4-3ffa-460e-9c4c-579273d98ec9 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:30.942120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d7be4-3ffa-460e-9c4c-579273d98ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:30.943478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d7be4-3ffa-460e-9c4c-579273d98ec9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:30.950195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 515/1026] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Sep 21 05:15:30 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:15:30.983500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:31.022885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587881bc-dead-4c09-9617-e05d420d9dfa None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:31.060865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190a013-e1f7-4a41-9c14-765e3bf0de46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:31.061495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a013-e1f7-4a41-9c14-765e3bf0de46 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:31.061776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a013-e1f7-4a41-9c14-765e3bf0de46 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:31.062082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a013-e1f7-4a41-9c14-765e3bf0de46 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:31.062747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190a013-e1f7-4a41-9c14-765e3bf0de46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:31.064192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190a013-e1f7-4a41-9c14-765e3bf0de46 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:31.097977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 512/1027] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:15:30 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:15:33.001084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf73ca-4c61-4985-b021-97c75d8c2903 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.001588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf73ca-4c61-4985-b021-97c75d8c2903 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.001588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf73ca-4c61-4985-b021-97c75d8c2903 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.002951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 516/1028] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:33.009071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac5615-676f-4e56-95c4-fb4d50492749 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.009598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac5615-676f-4e56-95c4-fb4d50492749 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.009598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac5615-676f-4e56-95c4-fb4d50492749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.029081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ecac5615-676f-4e56-95c4-fb4d50492749 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:33.030339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ecac5615-676f-4e56-95c4-fb4d50492749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:33.097929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 513/1029] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:33.115012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580e932-850e-4822-8b4b-a1139f65f861 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.115297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580e932-850e-4822-8b4b-a1139f65f861 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.115565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580e932-850e-4822-8b4b-a1139f65f861 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.117019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 517/1030] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:33.122712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fa512-aa60-499e-99cb-86ab05072942 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.123003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fa512-aa60-499e-99cb-86ab05072942 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.123278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fa512-aa60-499e-99cb-86ab05072942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.140907 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-455fa512-aa60-499e-99cb-86ab05072942 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:33.142108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-455fa512-aa60-499e-99cb-86ab05072942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:33.189572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:33.218006 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 514/1031] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:33.300837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038b97d-f76c-46b7-937b-cfd33016466c None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.301111 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038b97d-f76c-46b7-937b-cfd33016466c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.301315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038b97d-f76c-46b7-937b-cfd33016466c None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.302888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 518/1032] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:33.308344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea090d63-0fd4-4a78-93e6-3af0393b1dc5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.308632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea090d63-0fd4-4a78-93e6-3af0393b1dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.308907 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea090d63-0fd4-4a78-93e6-3af0393b1dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.326717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ea090d63-0fd4-4a78-93e6-3af0393b1dc5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:33.327897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ea090d63-0fd4-4a78-93e6-3af0393b1dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:33.372007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038b97d-f76c-46b7-937b-cfd33016466c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:33.398416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 515/1033] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:33.481286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038b97d-f76c-46b7-937b-cfd33016466c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:33.486869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea090d63-0fd4-4a78-93e6-3af0393b1dc5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:33.521041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b9ecd-e353-4625-ae7e-afbb48c05018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:33.521675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b9ecd-e353-4625-ae7e-afbb48c05018 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.521967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b9ecd-e353-4625-ae7e-afbb48c05018 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.522251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b9ecd-e353-4625-ae7e-afbb48c05018 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.530538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:33.534178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b9ecd-e353-4625-ae7e-afbb48c05018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3d6b9ecd-e353-4625-ae7e-afbb48c05018 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:33.539419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:33.540407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:33.542640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 519/1034] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Sep 21 05:15:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:33.547025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b9ecd-e353-4625-ae7e-afbb48c05018 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:33.609043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea090d63-0fd4-4a78-93e6-3af0393b1dc5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:33.620326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:33.647238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d8cf9-531f-41c3-85c0-649c0af28bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:33.647872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d8cf9-531f-41c3-85c0-649c0af28bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.648155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d8cf9-531f-41c3-85c0-649c0af28bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.648433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d8cf9-531f-41c3-85c0-649c0af28bb4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.649014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d8cf9-531f-41c3-85c0-649c0af28bb4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:33.651699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d8cf9-531f-41c3-85c0-649c0af28bb4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:33.659358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b9ecd-e353-4625-ae7e-afbb48c05018 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c84d8cf9-531f-41c3-85c0-649c0af28bb4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:33.660450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:33.661380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:33.662950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 516/1035] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Sep 21 05:15:33 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:33.670903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d8cf9-531f-41c3-85c0-649c0af28bb4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:33.696591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20981359-f507-4096-912c-4a94e9d6d79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:33.697195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20981359-f507-4096-912c-4a94e9d6d79d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.697479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20981359-f507-4096-912c-4a94e9d6d79d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.697777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20981359-f507-4096-912c-4a94e9d6d79d None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.698461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20981359-f507-4096-912c-4a94e9d6d79d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:33.698763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20981359-f507-4096-912c-4a94e9d6d79d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:33.700115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20981359-f507-4096-912c-4a94e9d6d79d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:33.707987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 520/1036] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Sep 21 05:15:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:33.708767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20981359-f507-4096-912c-4a94e9d6d79d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:33.770162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d8cf9-531f-41c3-85c0-649c0af28bb4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:33.770767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:33.803848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c74cb-db23-48fc-9abe-b26747146e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:33.804766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c74cb-db23-48fc-9abe-b26747146e4d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.805184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c74cb-db23-48fc-9abe-b26747146e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.805462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c74cb-db23-48fc-9abe-b26747146e4d None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.806106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c74cb-db23-48fc-9abe-b26747146e4d None admin] RBAC: Adding query filter params (name=image) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:33.806410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c74cb-db23-48fc-9abe-b26747146e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:33.807781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c74cb-db23-48fc-9abe-b26747146e4d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:33.809753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20981359-f507-4096-912c-4a94e9d6d79d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:33.812408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 517/1037] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:33.813162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c74cb-db23-48fc-9abe-b26747146e4d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:33.843941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47841-9b07-46c6-b5fd-7be3a609a484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:33.844499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47841-9b07-46c6-b5fd-7be3a609a484 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.844781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47841-9b07-46c6-b5fd-7be3a609a484 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.845055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47841-9b07-46c6-b5fd-7be3a609a484 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.846184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47841-9b07-46c6-b5fd-7be3a609a484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:33.847605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47841-9b07-46c6-b5fd-7be3a609a484 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:33.849158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 521/1038] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:33.849910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47841-9b07-46c6-b5fd-7be3a609a484 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:33.855422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:33.916795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c74cb-db23-48fc-9abe-b26747146e4d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:33.946476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47841-9b07-46c6-b5fd-7be3a609a484 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:33.949805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aeb58c-b3f9-4665-afc0-237a036ece82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:33.950467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeb58c-b3f9-4665-afc0-237a036ece82 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.950749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeb58c-b3f9-4665-afc0-237a036ece82 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.951020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeb58c-b3f9-4665-afc0-237a036ece82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.961236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aeb58c-b3f9-4665-afc0-237a036ece82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:33.962677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aeb58c-b3f9-4665-afc0-237a036ece82 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:33.974539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-50aeb58c-b3f9-4665-afc0-237a036ece82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:33.976752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:33.977693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:33.978604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:33.978604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:33.978604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:33.978604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 518/1039] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:33.982038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e2394-30a8-4355-b429-32e9d5e44bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:33.982795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e2394-30a8-4355-b429-32e9d5e44bbb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:33.982911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aeb58c-b3f9-4665-afc0-237a036ece82 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:33.983017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e2394-30a8-4355-b429-32e9d5e44bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:33.983183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e2394-30a8-4355-b429-32e9d5e44bbb None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:33.983805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e2394-30a8-4355-b429-32e9d5e44bbb None admin] RBAC: Adding query filter params (type=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:33.984109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e2394-30a8-4355-b429-32e9d5e44bbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:33.985504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e2394-30a8-4355-b429-32e9d5e44bbb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:33.990717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 522/1040] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:33.991485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e2394-30a8-4355-b429-32e9d5e44bbb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:34.075377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:34.084845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aeb58c-b3f9-4665-afc0-237a036ece82 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:34.087823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e2394-30a8-4355-b429-32e9d5e44bbb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:34.118356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28993ed3-95d8-4bfb-9c9e-7e6aae94cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:34.118895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28993ed3-95d8-4bfb-9c9e-7e6aae94cc2e None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:34.119176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28993ed3-95d8-4bfb-9c9e-7e6aae94cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:34.119500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28993ed3-95d8-4bfb-9c9e-7e6aae94cc2e None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:34.120114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28993ed3-95d8-4bfb-9c9e-7e6aae94cc2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:34.120663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb06148-a65d-4991-9180-711f00a5c890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:34.121265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb06148-a65d-4991-9180-711f00a5c890 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:34.121520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28993ed3-95d8-4bfb-9c9e-7e6aae94cc2e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:34.121623 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb06148-a65d-4991-9180-711f00a5c890 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:34.121807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb06148-a65d-4991-9180-711f00a5c890 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:34.122494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb06148-a65d-4991-9180-711f00a5c890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:34.122798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb06148-a65d-4991-9180-711f00a5c890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:34.124416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb06148-a65d-4991-9180-711f00a5c890 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:34.137109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 523/1041] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:34.138002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb06148-a65d-4991-9180-711f00a5c890 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:34.147478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 519/1042] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:15:33 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:34.148173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28993ed3-95d8-4bfb-9c9e-7e6aae94cc2e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:34.209434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:34.262145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb06148-a65d-4991-9180-711f00a5c890 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:34.280319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28993ed3-95d8-4bfb-9c9e-7e6aae94cc2e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:34.300665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188e0bd-a203-4ffb-942a-9745b88b4831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:34.301288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188e0bd-a203-4ffb-942a-9745b88b4831 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:34.301571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188e0bd-a203-4ffb-942a-9745b88b4831 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:34.301848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188e0bd-a203-4ffb-942a-9745b88b4831 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:34.302557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188e0bd-a203-4ffb-942a-9745b88b4831 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:34.303963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188e0bd-a203-4ffb-942a-9745b88b4831 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:34.320666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f1206-b4c7-445f-bdb7-b8de18dd5d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:34.321311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f1206-b4c7-445f-bdb7-b8de18dd5d84 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:34.321589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f1206-b4c7-445f-bdb7-b8de18dd5d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:34.321869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f1206-b4c7-445f-bdb7-b8de18dd5d84 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:34.327866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 524/1043] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:15:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:34.330917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f1206-b4c7-445f-bdb7-b8de18dd5d84 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:34.332389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f1206-b4c7-445f-bdb7-b8de18dd5d84 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-528f1206-b4c7-445f-bdb7-b8de18dd5d84 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:34.344900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:34.346035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:34.347438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 520/1044] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Sep 21 05:15:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:34.352428 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188e0bd-a203-4ffb-942a-9745b88b4831 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:34.473399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188e0bd-a203-4ffb-942a-9745b88b4831 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:34.481316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:34.513891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae29f73-e777-4693-9704-7096e7868754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:34.514575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae29f73-e777-4693-9704-7096e7868754 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:34.514854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae29f73-e777-4693-9704-7096e7868754 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:34.515126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae29f73-e777-4693-9704-7096e7868754 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:34.515852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae29f73-e777-4693-9704-7096e7868754 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:34.516157 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae29f73-e777-4693-9704-7096e7868754 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:34.517570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae29f73-e777-4693-9704-7096e7868754 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:34.535755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 525/1045] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Sep 21 05:15:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:34.541643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f1206-b4c7-445f-bdb7-b8de18dd5d84 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:34.557974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:34.662737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f1206-b4c7-445f-bdb7-b8de18dd5d84 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:34.696538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2444b50-dea6-4cd5-a398-6283cb4d2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:34.697146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2444b50-dea6-4cd5-a398-6283cb4d2b76 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:34.697421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2444b50-dea6-4cd5-a398-6283cb4d2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:34.697717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2444b50-dea6-4cd5-a398-6283cb4d2b76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:34.765344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2444b50-dea6-4cd5-a398-6283cb4d2b76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:34.766717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2444b50-dea6-4cd5-a398-6283cb4d2b76 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:34.774446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:34.869028 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 521/1046] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 05:15:34.875238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae29f73-e777-4693-9704-7096e7868754 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:34.985511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae29f73-e777-4693-9704-7096e7868754 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:35.022856 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c4ab3-2c17-4247-9bfd-957a892199f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:35.023844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c4ab3-2c17-4247-9bfd-957a892199f6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:35.024390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c4ab3-2c17-4247-9bfd-957a892199f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:35.024942 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c4ab3-2c17-4247-9bfd-957a892199f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:35.036700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c4ab3-2c17-4247-9bfd-957a892199f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:35.038489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c4ab3-2c17-4247-9bfd-957a892199f6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-784c4ab3-2c17-4247-9bfd-957a892199f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:35.054859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:35.055961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:35.056860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:35.056860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:35.056860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:35.059079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 526/1047] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:35.065966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2444b50-dea6-4cd5-a398-6283cb4d2b76 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:35.176123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2444b50-dea6-4cd5-a398-6283cb4d2b76 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:35.184943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:35.213091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fee816-04dc-48f2-a91c-979bec86b318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:35.214669 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fee816-04dc-48f2-a91c-979bec86b318 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:35.214985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fee816-04dc-48f2-a91c-979bec86b318 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:35.215223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fee816-04dc-48f2-a91c-979bec86b318 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:35.215893 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fee816-04dc-48f2-a91c-979bec86b318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:35.216248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fee816-04dc-48f2-a91c-979bec86b318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:35.217595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fee816-04dc-48f2-a91c-979bec86b318 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:35.235091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 522/1048] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:35.236535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c4ab3-2c17-4247-9bfd-957a892199f6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:35.258951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:35.343425 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c4ab3-2c17-4247-9bfd-957a892199f6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:35.377446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ba23-8253-4627-87fa-a0c68c9e768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:35.378100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ba23-8253-4627-87fa-a0c68c9e768f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:35.378378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ba23-8253-4627-87fa-a0c68c9e768f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:35.378646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ba23-8253-4627-87fa-a0c68c9e768f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:35.385732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9ba23-8253-4627-87fa-a0c68c9e768f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:35.387202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9ba23-8253-4627-87fa-a0c68c9e768f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-98b9ba23-8253-4627-87fa-a0c68c9e768f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:35.399434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:35.400454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:35.402490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:35.402490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:35.402490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:35.402490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:35.402490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:35.402490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:35.402490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:35.402490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:35.406291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 527/1049] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:35.408493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fee816-04dc-48f2-a91c-979bec86b318 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:35.519742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fee816-04dc-48f2-a91c-979bec86b318 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:35.553861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a880c-1763-48e1-8c6f-6058bfa84b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:35.554616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a880c-1763-48e1-8c6f-6058bfa84b6b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:35.554994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a880c-1763-48e1-8c6f-6058bfa84b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:35.555364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a880c-1763-48e1-8c6f-6058bfa84b6b None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:35.556140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a880c-1763-48e1-8c6f-6058bfa84b6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:35.556560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a880c-1763-48e1-8c6f-6058bfa84b6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:35.558132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a880c-1763-48e1-8c6f-6058bfa84b6b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:35.570758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 523/1050] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:35.577251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ba23-8253-4627-87fa-a0c68c9e768f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:35.663324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:35.687169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ba23-8253-4627-87fa-a0c68c9e768f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:35.720884 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab11c0-3a0a-4b9b-9cfc-1743c309416a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:35.721507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab11c0-3a0a-4b9b-9cfc-1743c309416a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:35.721786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab11c0-3a0a-4b9b-9cfc-1743c309416a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:35.722093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab11c0-3a0a-4b9b-9cfc-1743c309416a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:35.722807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab11c0-3a0a-4b9b-9cfc-1743c309416a None admin] RBAC: Adding query filter params (role.id=688834a9cd994fa08909c0296f33b5a0, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d9168d8bbf8b43299a7748c9faa099b1) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:35.723117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab11c0-3a0a-4b9b-9cfc-1743c309416a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:35.724534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab11c0-3a0a-4b9b-9cfc-1743c309416a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:35.729970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 528/1051] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:35 2022] GET /identity/v3/role_assignments?user.id=d9168d8bbf8b43299a7748c9faa099b1&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=688834a9cd994fa08909c0296f33b5a0 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:36.460244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5efcd-3ec2-4119-9a4b-674d7d42231f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:36.460773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5efcd-3ec2-4119-9a4b-674d7d42231f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:36.460773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5efcd-3ec2-4119-9a4b-674d7d42231f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:36.462414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 524/1052] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:36.468197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2427d1a-ace0-4603-842a-bca3134b6ba4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:36.468635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2427d1a-ace0-4603-842a-bca3134b6ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:36.468837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2427d1a-ace0-4603-842a-bca3134b6ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:36.487811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d2427d1a-ace0-4603-842a-bca3134b6ba4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:36.489003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d2427d1a-ace0-4603-842a-bca3134b6ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:36.532165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd474b-9e32-43e9-bcd2-c0098dbd570a None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:36.532780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd474b-9e32-43e9-bcd2-c0098dbd570a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:36.532780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd474b-9e32-43e9-bcd2-c0098dbd570a None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:36.534131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 525/1053] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:36.539338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de682e4-afe8-4624-9384-4ae56b4558fd None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:36.539626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de682e4-afe8-4624-9384-4ae56b4558fd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:36.539904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de682e4-afe8-4624-9384-4ae56b4558fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:36.557969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 529/1054] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:36.559350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8de682e4-afe8-4624-9384-4ae56b4558fd None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:36.560556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8de682e4-afe8-4624-9384-4ae56b4558fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:36.583717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:36.641080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 526/1055] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:36.832451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2427d1a-ace0-4603-842a-bca3134b6ba4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:36.902901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:36.910372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de682e4-afe8-4624-9384-4ae56b4558fd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:36.935709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2427d1a-ace0-4603-842a-bca3134b6ba4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:36.972447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b83cc0-6f95-4cf6-ac08-b81ee4e51572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:36.973410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b83cc0-6f95-4cf6-ac08-b81ee4e51572 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:36.973875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b83cc0-6f95-4cf6-ac08-b81ee4e51572 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:36.974360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b83cc0-6f95-4cf6-ac08-b81ee4e51572 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:36.987926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b83cc0-6f95-4cf6-ac08-b81ee4e51572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-30b83cc0-6f95-4cf6-ac08-b81ee4e51572 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:36.994051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:36.995004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:36.995832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:36.996891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 530/1056] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:15:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:37.001394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b83cc0-6f95-4cf6-ac08-b81ee4e51572 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:37.018412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de682e4-afe8-4624-9384-4ae56b4558fd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:37.056132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902d8dc-4966-40cc-99e7-1c226cfc7c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:37.056732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902d8dc-4966-40cc-99e7-1c226cfc7c75 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.057050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902d8dc-4966-40cc-99e7-1c226cfc7c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.057321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902d8dc-4966-40cc-99e7-1c226cfc7c75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.083042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:37.108336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b83cc0-6f95-4cf6-ac08-b81ee4e51572 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:37.127354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902d8dc-4966-40cc-99e7-1c226cfc7c75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:37.128778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902d8dc-4966-40cc-99e7-1c226cfc7c75 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:37.139091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:37.145431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a05a-2f38-4821-912e-20143d4ebdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:37.146050 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a05a-2f38-4821-912e-20143d4ebdaf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.146329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a05a-2f38-4821-912e-20143d4ebdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.146599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a05a-2f38-4821-912e-20143d4ebdaf None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.147240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3a05a-2f38-4821-912e-20143d4ebdaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:37.147545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3a05a-2f38-4821-912e-20143d4ebdaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:37.148893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3a05a-2f38-4821-912e-20143d4ebdaf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:37.158819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 531/1057] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:15:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:37.164505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a05a-2f38-4821-912e-20143d4ebdaf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:37.233977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 527/1058] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 05:15:37.238795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902d8dc-4966-40cc-99e7-1c226cfc7c75 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:37.271308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a05a-2f38-4821-912e-20143d4ebdaf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:37.297369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:37.305280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b42d09-4965-44b5-bf16-8f736329ddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:37.305918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b42d09-4965-44b5-bf16-8f736329ddfa None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.306242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b42d09-4965-44b5-bf16-8f736329ddfa None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.306501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b42d09-4965-44b5-bf16-8f736329ddfa None admin] PATH_INFO: `/v3/users/7e7c384e4153465b986697afba525773` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.321501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b42d09-4965-44b5-bf16-8f736329ddfa None admin] RBAC: Authorizing `identity:get_user(user_id=7e7c384e4153465b986697afba525773)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:37.323557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b42d09-4965-44b5-bf16-8f736329ddfa None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:37.325366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 532/1059] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:15:37 2022] GET /identity/v3/users/7e7c384e4153465b986697afba525773 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:37.330832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b42d09-4965-44b5-bf16-8f736329ddfa None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:37.345679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902d8dc-4966-40cc-99e7-1c226cfc7c75 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:37.381009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:37.383741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f4fd6-68fa-40b4-a4fc-573c090d378e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:37.384371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f4fd6-68fa-40b4-a4fc-573c090d378e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.384934 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f4fd6-68fa-40b4-a4fc-573c090d378e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.385789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f4fd6-68fa-40b4-a4fc-573c090d378e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.396921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f4fd6-68fa-40b4-a4fc-573c090d378e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:37.398405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f4fd6-68fa-40b4-a4fc-573c090d378e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-6f6f4fd6-68fa-40b4-a4fc-573c090d378e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:37.414593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:37.415551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:37.416487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:37.416487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:37.416487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:37.418432 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 528/1060] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:37.422568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f4fd6-68fa-40b4-a4fc-573c090d378e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:37.434238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b42d09-4965-44b5-bf16-8f736329ddfa None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:37.469348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35110700-c0bf-454a-8701-d1a0a5a0f6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:37.470002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35110700-c0bf-454a-8701-d1a0a5a0f6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.470242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35110700-c0bf-454a-8701-d1a0a5a0f6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.470506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35110700-c0bf-454a-8701-d1a0a5a0f6e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.532864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f4fd6-68fa-40b4-a4fc-573c090d378e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:37.542426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:37.546666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35110700-c0bf-454a-8701-d1a0a5a0f6e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:37.548129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35110700-c0bf-454a-8701-d1a0a5a0f6e2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:37.571846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef4e1-0ad3-4514-9e52-74e8202712f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:37.573499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef4e1-0ad3-4514-9e52-74e8202712f7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.574055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef4e1-0ad3-4514-9e52-74e8202712f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.574462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef4e1-0ad3-4514-9e52-74e8202712f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.575355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cef4e1-0ad3-4514-9e52-74e8202712f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:37.575799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cef4e1-0ad3-4514-9e52-74e8202712f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:37.578130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cef4e1-0ad3-4514-9e52-74e8202712f7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:37.595279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 529/1061] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:37.602982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:37.667063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 533/1062] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 05:15:37.668804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef4e1-0ad3-4514-9e52-74e8202712f7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:37.799063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef4e1-0ad3-4514-9e52-74e8202712f7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:37.834860 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17910a6a-a20e-41af-a0ea-ab5ddfe90ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:37.835495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17910a6a-a20e-41af-a0ea-ab5ddfe90ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.835775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17910a6a-a20e-41af-a0ea-ab5ddfe90ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.836043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17910a6a-a20e-41af-a0ea-ab5ddfe90ed5 None admin] PATH_INFO: `/v3/projects/97051a34541c485eba1b2aab1fd0eb5d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.837157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17910a6a-a20e-41af-a0ea-ab5ddfe90ed5 None admin] RBAC: Authorizing `identity:get_project(project_id=97051a34541c485eba1b2aab1fd0eb5d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:37.838645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17910a6a-a20e-41af-a0ea-ab5ddfe90ed5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:37.840437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 530/1063] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:15:37 2022] GET /identity/v3/projects/97051a34541c485eba1b2aab1fd0eb5d => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:37.846566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35110700-c0bf-454a-8701-d1a0a5a0f6e2 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:37.868380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4337-1ba8-48a8-886c-cca7cf5c67c8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.868572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4337-1ba8-48a8-886c-cca7cf5c67c8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.868845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4337-1ba8-48a8-886c-cca7cf5c67c8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.870925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 531/1064] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:37.875731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09e317-5f7f-4195-ad50-ad223c975323 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.876009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09e317-5f7f-4195-ad50-ad223c975323 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.876279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09e317-5f7f-4195-ad50-ad223c975323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.895676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1a09e317-5f7f-4195-ad50-ad223c975323 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:37.896855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1a09e317-5f7f-4195-ad50-ad223c975323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:37.929478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:37.958950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35110700-c0bf-454a-8701-d1a0a5a0f6e2 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:37.970051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 532/1065] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:37.993634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29c63b-4add-4c48-b7c8-fdc7da9bc389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:37.994267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29c63b-4add-4c48-b7c8-fdc7da9bc389 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:37.994597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29c63b-4add-4c48-b7c8-fdc7da9bc389 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:37.994867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29c63b-4add-4c48-b7c8-fdc7da9bc389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:37.995579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29c63b-4add-4c48-b7c8-fdc7da9bc389 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=97051a34541c485eba1b2aab1fd0eb5d, user.id=7e7c384e4153465b986697afba525773) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:37.995883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29c63b-4add-4c48-b7c8-fdc7da9bc389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:37.997293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29c63b-4add-4c48-b7c8-fdc7da9bc389 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:38.013962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 534/1066] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:37 2022] GET /identity/v3/role_assignments?user.id=7e7c384e4153465b986697afba525773&scope.project.id=97051a34541c485eba1b2aab1fd0eb5d&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:38.238306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09e317-5f7f-4195-ad50-ad223c975323 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:38.333471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09e317-5f7f-4195-ad50-ad223c975323 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:38.366678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfcb1df-4c50-4268-bedd-d7ea71105d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:38.367311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcb1df-4c50-4268-bedd-d7ea71105d42 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:38.367590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcb1df-4c50-4268-bedd-d7ea71105d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:38.367862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcb1df-4c50-4268-bedd-d7ea71105d42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:38.377549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcb1df-4c50-4268-bedd-d7ea71105d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5cfcb1df-4c50-4268-bedd-d7ea71105d42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:38.382331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:38.383423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:38.384283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 533/1067] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Sep 21 05:15:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:38.389827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29c63b-4add-4c48-b7c8-fdc7da9bc389 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:38.406070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:38.488282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29c63b-4add-4c48-b7c8-fdc7da9bc389 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:38.521204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab19b1b-e61b-40a5-833f-243b49de8ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:38.521735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab19b1b-e61b-40a5-833f-243b49de8ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:38.522042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab19b1b-e61b-40a5-833f-243b49de8ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:38.522316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab19b1b-e61b-40a5-833f-243b49de8ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:38.522965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab19b1b-e61b-40a5-833f-243b49de8ab5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:38.523267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab19b1b-e61b-40a5-833f-243b49de8ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:38.524694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab19b1b-e61b-40a5-833f-243b49de8ab5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:38.532521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 535/1068] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Sep 21 05:15:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:38.538217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfcb1df-4c50-4268-bedd-d7ea71105d42 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:38.567418 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:38.646689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfcb1df-4c50-4268-bedd-d7ea71105d42 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:38.679438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bd73b-ca5a-4bd2-858f-7cd62b080377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:38.680063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bd73b-ca5a-4bd2-858f-7cd62b080377 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:38.680340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bd73b-ca5a-4bd2-858f-7cd62b080377 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:38.680609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bd73b-ca5a-4bd2-858f-7cd62b080377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:38.691603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bd73b-ca5a-4bd2-858f-7cd62b080377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:38.693032 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bd73b-ca5a-4bd2-858f-7cd62b080377 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d61bd73b-ca5a-4bd2-858f-7cd62b080377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:38.706775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:38.707990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:38.709117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:38.709117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:38.709117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:38.710089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 534/1069] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:38.714061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab19b1b-e61b-40a5-833f-243b49de8ab5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:38.792026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:38.816449 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab19b1b-e61b-40a5-833f-243b49de8ab5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:38.848973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4731d6-30c1-4e74-9c4e-e849f806cda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:38.849596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4731d6-30c1-4e74-9c4e-e849f806cda9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:38.849873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4731d6-30c1-4e74-9c4e-e849f806cda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:38.850174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4731d6-30c1-4e74-9c4e-e849f806cda9 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:38.850829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4731d6-30c1-4e74-9c4e-e849f806cda9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:38.851133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4731d6-30c1-4e74-9c4e-e849f806cda9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:38.852544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4731d6-30c1-4e74-9c4e-e849f806cda9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:38.866396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 536/1070] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:38.872017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bd73b-ca5a-4bd2-858f-7cd62b080377 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:38.947913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:38.969154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bd73b-ca5a-4bd2-858f-7cd62b080377 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:39.001482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b01a28-64cc-4c86-9f26-6865801e4fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:39.002127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b01a28-64cc-4c86-9f26-6865801e4fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:39.002407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b01a28-64cc-4c86-9f26-6865801e4fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:39.002677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b01a28-64cc-4c86-9f26-6865801e4fa2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:39.009733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b01a28-64cc-4c86-9f26-6865801e4fa2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:39.011217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b01a28-64cc-4c86-9f26-6865801e4fa2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d2b01a28-64cc-4c86-9f26-6865801e4fa2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:39.021922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:39.023077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:15:39.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:39.026256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 535/1071] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Sep 21 05:15:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:39.029117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4731d6-30c1-4e74-9c4e-e849f806cda9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:39.126104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4731d6-30c1-4e74-9c4e-e849f806cda9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:39.160085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d1880-c305-4793-a470-57cb6a01059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:39.160683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d1880-c305-4793-a470-57cb6a01059e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:39.160964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d1880-c305-4793-a470-57cb6a01059e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:39.161238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d1880-c305-4793-a470-57cb6a01059e None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:39.161975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d1880-c305-4793-a470-57cb6a01059e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:39.162288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d1880-c305-4793-a470-57cb6a01059e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:39.163711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d1880-c305-4793-a470-57cb6a01059e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:39.166307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:39.181748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 537/1072] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Sep 21 05:15:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:39.187417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b01a28-64cc-4c86-9f26-6865801e4fa2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:39.285051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b01a28-64cc-4c86-9f26-6865801e4fa2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:39.316402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46189934-5485-4885-91f4-42e772d255fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:39.317016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46189934-5485-4885-91f4-42e772d255fc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:39.317294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46189934-5485-4885-91f4-42e772d255fc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:39.317563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46189934-5485-4885-91f4-42e772d255fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:39.325100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:39.329513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46189934-5485-4885-91f4-42e772d255fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:39.330956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46189934-5485-4885-91f4-42e772d255fc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-46189934-5485-4885-91f4-42e772d255fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:39.345048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:39.346233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:39.347158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:39.347158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:39.347158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:39.347158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 536/1073] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:39.351745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d1880-c305-4793-a470-57cb6a01059e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:39.448829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d1880-c305-4793-a470-57cb6a01059e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:39.482473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd040a5c-b376-47e6-8c52-b4f8e144f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:39.483070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd040a5c-b376-47e6-8c52-b4f8e144f9fc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:39.483348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd040a5c-b376-47e6-8c52-b4f8e144f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:39.483622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd040a5c-b376-47e6-8c52-b4f8e144f9fc None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:39.484251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd040a5c-b376-47e6-8c52-b4f8e144f9fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:39.484555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd040a5c-b376-47e6-8c52-b4f8e144f9fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:39.485907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd040a5c-b376-47e6-8c52-b4f8e144f9fc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:39.497770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 538/1074] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:39.503425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46189934-5485-4885-91f4-42e772d255fc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:39.600951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46189934-5485-4885-91f4-42e772d255fc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:39.634049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703079a-eb50-4139-a776-0923a7d3601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:39.634671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703079a-eb50-4139-a776-0923a7d3601f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:39.634948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703079a-eb50-4139-a776-0923a7d3601f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:39.635830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703079a-eb50-4139-a776-0923a7d3601f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:39.642359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703079a-eb50-4139-a776-0923a7d3601f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:39.643764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703079a-eb50-4139-a776-0923a7d3601f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a703079a-eb50-4139-a776-0923a7d3601f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:39.654626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:39.660321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:39.661333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:39.661333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:39.661333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:39.661333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:39.661333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:39.661333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:39.661333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:39.661333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:39.661333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 537/1075] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:39.661845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd040a5c-b376-47e6-8c52-b4f8e144f9fc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:39.715628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:39.760851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd040a5c-b376-47e6-8c52-b4f8e144f9fc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:39.793865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebf440-f9c1-4b6b-a8a4-48ac211f8dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:39.794495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebf440-f9c1-4b6b-a8a4-48ac211f8dff None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:39.794776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebf440-f9c1-4b6b-a8a4-48ac211f8dff None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:39.795047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebf440-f9c1-4b6b-a8a4-48ac211f8dff None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:39.795726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebf440-f9c1-4b6b-a8a4-48ac211f8dff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:39.796032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebf440-f9c1-4b6b-a8a4-48ac211f8dff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:39.797455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebf440-f9c1-4b6b-a8a4-48ac211f8dff None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:39.809984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 539/1076] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:39.817449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703079a-eb50-4139-a776-0923a7d3601f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:39.867574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:39.915654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703079a-eb50-4139-a776-0923a7d3601f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:39.948223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87406c-e896-4a43-8a34-61b006e56250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:39.948874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87406c-e896-4a43-8a34-61b006e56250 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:39.949154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87406c-e896-4a43-8a34-61b006e56250 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:39.949426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87406c-e896-4a43-8a34-61b006e56250 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d9168d8bbf8b43299a7748c9faa099b1/roles/688834a9cd994fa08909c0296f33b5a0` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:39.951530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87406c-e896-4a43-8a34-61b006e56250 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=d9168d8bbf8b43299a7748c9faa099b1, role_id=688834a9cd994fa08909c0296f33b5a0)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:39.952959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87406c-e896-4a43-8a34-61b006e56250 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:39.965352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 538/1077] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:15:39 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/d9168d8bbf8b43299a7748c9faa099b1/roles/688834a9cd994fa08909c0296f33b5a0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:15:40.144756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7b6cf-46f9-4c0b-aeeb-58d78c97ad3e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:40.145046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7b6cf-46f9-4c0b-aeeb-58d78c97ad3e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:40.145320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7b6cf-46f9-4c0b-aeeb-58d78c97ad3e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:40.146731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 540/1078] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:40.152018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13c7b7-f184-422f-ba00-7cb25688540a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:40.152302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13c7b7-f184-422f-ba00-7cb25688540a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:40.152580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13c7b7-f184-422f-ba00-7cb25688540a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:40.171391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-cf13c7b7-f184-422f-ba00-7cb25688540a None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:40.172562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cf13c7b7-f184-422f-ba00-7cb25688540a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:40.239163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 539/1079] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:40.507527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7b6cf-46f9-4c0b-aeeb-58d78c97ad3e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:40.611239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7b6cf-46f9-4c0b-aeeb-58d78c97ad3e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:40.625182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:40.645430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd230d0-db13-4fb8-9120-25b51dbe9d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:40.646102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd230d0-db13-4fb8-9120-25b51dbe9d39 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:40.646383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd230d0-db13-4fb8-9120-25b51dbe9d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:40.646657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd230d0-db13-4fb8-9120-25b51dbe9d39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:40.657081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd230d0-db13-4fb8-9120-25b51dbe9d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-7bd230d0-db13-4fb8-9120-25b51dbe9d39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:40.661832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:40.662984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:40.664019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:40.666078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 541/1080] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:15:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:40.668951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13c7b7-f184-422f-ba00-7cb25688540a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:40.708734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:40.764872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13c7b7-f184-422f-ba00-7cb25688540a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:40.797615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f116-08e2-4db2-b60d-fd74335287ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:40.798254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f116-08e2-4db2-b60d-fd74335287ed None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:40.798525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f116-08e2-4db2-b60d-fd74335287ed None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:40.798796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f116-08e2-4db2-b60d-fd74335287ed None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:40.799445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5f116-08e2-4db2-b60d-fd74335287ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:40.799737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5f116-08e2-4db2-b60d-fd74335287ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:40.801081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5f116-08e2-4db2-b60d-fd74335287ed None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:40.809630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 540/1081] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:15:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:40.815469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd230d0-db13-4fb8-9120-25b51dbe9d39 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:40.915597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd230d0-db13-4fb8-9120-25b51dbe9d39 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:40.948227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5432925-2d6b-4fb9-96f5-7dcb76ab0bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:40.948839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5432925-2d6b-4fb9-96f5-7dcb76ab0bbe None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:40.949114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5432925-2d6b-4fb9-96f5-7dcb76ab0bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:40.949385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5432925-2d6b-4fb9-96f5-7dcb76ab0bbe None admin] PATH_INFO: `/v3/users/7e7c384e4153465b986697afba525773` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:40.950495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5432925-2d6b-4fb9-96f5-7dcb76ab0bbe None admin] RBAC: Authorizing `identity:get_user(user_id=7e7c384e4153465b986697afba525773)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:40.951906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5432925-2d6b-4fb9-96f5-7dcb76ab0bbe None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:40.953452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 542/1082] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:15:40 2022] GET /identity/v3/users/7e7c384e4153465b986697afba525773 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:40.959203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f116-08e2-4db2-b60d-fd74335287ed None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:41.053848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f116-08e2-4db2-b60d-fd74335287ed None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:41.055301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:41.088579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5161651-9b83-48aa-8728-11fa98500ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:41.089201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5161651-9b83-48aa-8728-11fa98500ccf None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:41.089481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5161651-9b83-48aa-8728-11fa98500ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:41.089754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5161651-9b83-48aa-8728-11fa98500ccf None admin] PATH_INFO: `/v3/projects/97051a34541c485eba1b2aab1fd0eb5d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:41.090882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5161651-9b83-48aa-8728-11fa98500ccf None admin] RBAC: Authorizing `identity:get_project(project_id=97051a34541c485eba1b2aab1fd0eb5d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:41.092292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5161651-9b83-48aa-8728-11fa98500ccf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:41.093900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 541/1083] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:15:40 2022] GET /identity/v3/projects/97051a34541c485eba1b2aab1fd0eb5d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:41.099710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5432925-2d6b-4fb9-96f5-7dcb76ab0bbe None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:41.141807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:41.195738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5432925-2d6b-4fb9-96f5-7dcb76ab0bbe None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:41.228713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39dbdf-e3fc-47ea-b2e9-ae689a6a505d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:41.229355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39dbdf-e3fc-47ea-b2e9-ae689a6a505d None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:41.229629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39dbdf-e3fc-47ea-b2e9-ae689a6a505d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:41.229899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39dbdf-e3fc-47ea-b2e9-ae689a6a505d None admin] PATH_INFO: `/v3/projects/97051a34541c485eba1b2aab1fd0eb5d/users/7e7c384e4153465b986697afba525773/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:41.231775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39dbdf-e3fc-47ea-b2e9-ae689a6a505d None admin] RBAC: Authorizing `identity:create_grant(project_id=97051a34541c485eba1b2aab1fd0eb5d, user_id=7e7c384e4153465b986697afba525773, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:41.233193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39dbdf-e3fc-47ea-b2e9-ae689a6a505d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:41.246011 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 543/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:15:41 2022] PUT /identity/v3/projects/97051a34541c485eba1b2aab1fd0eb5d/users/7e7c384e4153465b986697afba525773/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:15:42.119055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8b78-aaf0-45c1-b911-855ba9cb5165 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:42.119658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8b78-aaf0-45c1-b911-855ba9cb5165 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:42.119658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8b78-aaf0-45c1-b911-855ba9cb5165 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:42.120918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 542/1085] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:42.126301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23568d-f91d-4f03-8df6-73a3dae7cc29 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:42.126710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23568d-f91d-4f03-8df6-73a3dae7cc29 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:42.126838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23568d-f91d-4f03-8df6-73a3dae7cc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:42.144901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9b23568d-f91d-4f03-8df6-73a3dae7cc29 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:42.146090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9b23568d-f91d-4f03-8df6-73a3dae7cc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:42.210768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 544/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:42.476442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8b78-aaf0-45c1-b911-855ba9cb5165 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:42.575293 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8b78-aaf0-45c1-b911-855ba9cb5165 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:42.610656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb3ca8-424b-4488-b861-f868e3768587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:42.611286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb3ca8-424b-4488-b861-f868e3768587 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:42.611566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb3ca8-424b-4488-b861-f868e3768587 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:42.611838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb3ca8-424b-4488-b861-f868e3768587 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:42.621902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb3ca8-424b-4488-b861-f868e3768587 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-e9cb3ca8-424b-4488-b861-f868e3768587 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:42.626740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:42.627785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:42.628609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 543/1087] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Sep 21 05:15:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:42.633789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23568d-f91d-4f03-8df6-73a3dae7cc29 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:42.663019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:42.729601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23568d-f91d-4f03-8df6-73a3dae7cc29 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:42.768845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c88c6-75e8-498a-aee9-55cb3950a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:42.769443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c88c6-75e8-498a-aee9-55cb3950a23d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:42.769717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c88c6-75e8-498a-aee9-55cb3950a23d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:42.770012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c88c6-75e8-498a-aee9-55cb3950a23d None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:42.770663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c88c6-75e8-498a-aee9-55cb3950a23d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:42.770959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c88c6-75e8-498a-aee9-55cb3950a23d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:42.772316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c88c6-75e8-498a-aee9-55cb3950a23d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:42.780897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 545/1088] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Sep 21 05:15:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:42.786684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb3ca8-424b-4488-b861-f868e3768587 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:42.820305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:42.889835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb3ca8-424b-4488-b861-f868e3768587 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:42.925615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f08e20-465c-4941-8358-4b09cd5a6e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:42.926313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f08e20-465c-4941-8358-4b09cd5a6e6a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:42.926586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f08e20-465c-4941-8358-4b09cd5a6e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:42.926855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f08e20-465c-4941-8358-4b09cd5a6e6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:42.936651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f08e20-465c-4941-8358-4b09cd5a6e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:42.938104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f08e20-465c-4941-8358-4b09cd5a6e6a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-60f08e20-465c-4941-8358-4b09cd5a6e6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:42.952512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:42.953553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:42.954835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:42.954835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:42.954835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:42.954835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 544/1089] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:42.959264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c88c6-75e8-498a-aee9-55cb3950a23d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:43.050840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:43.057811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c88c6-75e8-498a-aee9-55cb3950a23d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:43.091439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850bf11-3cac-45f0-b425-84c3c3968aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:43.092037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850bf11-3cac-45f0-b425-84c3c3968aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:43.092312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850bf11-3cac-45f0-b425-84c3c3968aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:43.092581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850bf11-3cac-45f0-b425-84c3c3968aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:43.093208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850bf11-3cac-45f0-b425-84c3c3968aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:43.093506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850bf11-3cac-45f0-b425-84c3c3968aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:43.094896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850bf11-3cac-45f0-b425-84c3c3968aa2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:43.106566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 546/1090] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:43.112763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f08e20-465c-4941-8358-4b09cd5a6e6a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:43.190579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:43.208913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f08e20-465c-4941-8358-4b09cd5a6e6a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:43.241579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62dfed-b910-4af5-b83c-a61b71bf5503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:43.242228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62dfed-b910-4af5-b83c-a61b71bf5503 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:43.242505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62dfed-b910-4af5-b83c-a61b71bf5503 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:43.242772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62dfed-b910-4af5-b83c-a61b71bf5503 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:43.250427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62dfed-b910-4af5-b83c-a61b71bf5503 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:43.251850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62dfed-b910-4af5-b83c-a61b71bf5503 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-0b62dfed-b910-4af5-b83c-a61b71bf5503 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:15:43.263071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:43.264155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:43.265089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 545/1091] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Sep 21 05:15:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:43.269892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850bf11-3cac-45f0-b425-84c3c3968aa2 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:43.368817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850bf11-3cac-45f0-b425-84c3c3968aa2 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:43.402186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e6020-aa42-4379-aebb-01eb28dc0be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:43.402788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e6020-aa42-4379-aebb-01eb28dc0be3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:43.402903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e9870-453f-4aad-ac24-b981d1e9f308 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:43.403063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e6020-aa42-4379-aebb-01eb28dc0be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:43.403156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e9870-453f-4aad-ac24-b981d1e9f308 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:43.403329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e6020-aa42-4379-aebb-01eb28dc0be3 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:43.403429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e9870-453f-4aad-ac24-b981d1e9f308 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:43.404039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e6020-aa42-4379-aebb-01eb28dc0be3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:43.404339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e6020-aa42-4379-aebb-01eb28dc0be3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:43.404721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 546/1092] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:15:43.405775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e6020-aa42-4379-aebb-01eb28dc0be3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:43.409575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32b40f-3426-4a1a-b44e-9453641c089b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:43.409857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32b40f-3426-4a1a-b44e-9453641c089b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:43.410158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32b40f-3426-4a1a-b44e-9453641c089b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:43.422763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 547/1093] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Sep 21 05:15:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:43.428066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-cb32b40f-3426-4a1a-b44e-9453641c089b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:43.428568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e6020-aa42-4379-aebb-01eb28dc0be3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:43.429245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cb32b40f-3426-4a1a-b44e-9453641c089b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:43.443689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:43.495482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 547/1094] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:43.528555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e6020-aa42-4379-aebb-01eb28dc0be3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:43.563155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451fa2b-0066-4898-a23b-4c95148dc272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:43.563835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451fa2b-0066-4898-a23b-4c95148dc272 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:43.564113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451fa2b-0066-4898-a23b-4c95148dc272 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:43.564376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451fa2b-0066-4898-a23b-4c95148dc272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:43.633052 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451fa2b-0066-4898-a23b-4c95148dc272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:43.634473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451fa2b-0066-4898-a23b-4c95148dc272 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:43.648084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:43.729714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 548/1095] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 05:15:43.735266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32b40f-3426-4a1a-b44e-9453641c089b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:43.765574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451fa2b-0066-4898-a23b-4c95148dc272 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:43.804842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:43.833870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32b40f-3426-4a1a-b44e-9453641c089b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:43.862335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451fa2b-0066-4898-a23b-4c95148dc272 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:43.867492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aabf4b-83e7-4c09-b3f2-21ee7f4eb42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:43.868122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aabf4b-83e7-4c09-b3f2-21ee7f4eb42c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:43.868403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aabf4b-83e7-4c09-b3f2-21ee7f4eb42c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:43.868676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aabf4b-83e7-4c09-b3f2-21ee7f4eb42c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:43.876244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:43.879529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aabf4b-83e7-4c09-b3f2-21ee7f4eb42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:43.880969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aabf4b-83e7-4c09-b3f2-21ee7f4eb42c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-25aabf4b-83e7-4c09-b3f2-21ee7f4eb42c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:15:43.895281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:43.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:15:43.897531 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:15:43.897531 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:15:43.897531 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:43.897531 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 548/1096] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:15:43.898129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f3ac-564f-4bca-818f-0135147c75f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:43.898760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f3ac-564f-4bca-818f-0135147c75f3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:43.899041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f3ac-564f-4bca-818f-0135147c75f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:43.899316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f3ac-564f-4bca-818f-0135147c75f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:43.901859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aabf4b-83e7-4c09-b3f2-21ee7f4eb42c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:43.909362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9f3ac-564f-4bca-818f-0135147c75f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-57b9f3ac-564f-4bca-818f-0135147c75f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:15:43.914177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:43.918125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:15:43.919284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 549/1097] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:15:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:43.921485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f3ac-564f-4bca-818f-0135147c75f3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:44.005589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aabf4b-83e7-4c09-b3f2-21ee7f4eb42c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:44.018996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f3ac-564f-4bca-818f-0135147c75f3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:44.028368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:44.039245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374da4-d5a0-48b2-a201-a338df66fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:44.039899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374da4-d5a0-48b2-a201-a338df66fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:44.040177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374da4-d5a0-48b2-a201-a338df66fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:44.040446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374da4-d5a0-48b2-a201-a338df66fdd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:44.041097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00374da4-d5a0-48b2-a201-a338df66fdd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:44.041925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00374da4-d5a0-48b2-a201-a338df66fdd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:44.043290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00374da4-d5a0-48b2-a201-a338df66fdd1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:44.053773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29727ac4-c288-418d-8bd4-676479619b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:44.054397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29727ac4-c288-418d-8bd4-676479619b4a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:44.054677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29727ac4-c288-418d-8bd4-676479619b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:44.054784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 549/1098] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:44.055022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29727ac4-c288-418d-8bd4-676479619b4a None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:44.055682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29727ac4-c288-418d-8bd4-676479619b4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:44.056049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29727ac4-c288-418d-8bd4-676479619b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:44.057411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29727ac4-c288-418d-8bd4-676479619b4a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:44.060289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374da4-d5a0-48b2-a201-a338df66fdd1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:44.070504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 550/1099] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:15:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:44.071480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29727ac4-c288-418d-8bd4-676479619b4a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:44.105134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:44.169332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374da4-d5a0-48b2-a201-a338df66fdd1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:44.172586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29727ac4-c288-418d-8bd4-676479619b4a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:44.202183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf76d3-082b-49de-86c4-ba964aced317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:44.202809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf76d3-082b-49de-86c4-ba964aced317 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:44.203122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf76d3-082b-49de-86c4-ba964aced317 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:44.203356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf76d3-082b-49de-86c4-ba964aced317 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:44.205785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2545-8df0-4ae1-bdf5-4265e0cf5abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:44.206441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2545-8df0-4ae1-bdf5-4265e0cf5abc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:44.206722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2545-8df0-4ae1-bdf5-4265e0cf5abc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:44.206998 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2545-8df0-4ae1-bdf5-4265e0cf5abc None admin] PATH_INFO: `/v3/users/7e7c384e4153465b986697afba525773` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:44.208162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a2545-8df0-4ae1-bdf5-4265e0cf5abc None admin] RBAC: Authorizing `identity:get_user(user_id=7e7c384e4153465b986697afba525773)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:44.209666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a2545-8df0-4ae1-bdf5-4265e0cf5abc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:44.210174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf76d3-082b-49de-86c4-ba964aced317 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:44.211312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 551/1100] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:15:44 2022] GET /identity/v3/users/7e7c384e4153465b986697afba525773 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:44.211666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf76d3-082b-49de-86c4-ba964aced317 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:44.217470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2545-8df0-4ae1-bdf5-4265e0cf5abc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-7aaf76d3-082b-49de-86c4-ba964aced317 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:15:44.223371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:15:44.224442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:15:44.225353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:15:44.225353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:15:44.225353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:15:44.225353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:15:44.225353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:15:44.225353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:15:44.225353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:15:44.225353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:15:44.225353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 550/1101] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:15:44.230888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf76d3-082b-49de-86c4-ba964aced317 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:44.274924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:44.316064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2545-8df0-4ae1-bdf5-4265e0cf5abc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:44.317551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:44.330153 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf76d3-082b-49de-86c4-ba964aced317 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:44.352100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b44017-30b0-4204-9de5-26e61bdc65c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:44.352701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b44017-30b0-4204-9de5-26e61bdc65c4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:44.352986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b44017-30b0-4204-9de5-26e61bdc65c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:44.353261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b44017-30b0-4204-9de5-26e61bdc65c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:44.364984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861e9ed-b20d-4d41-81e2-66c274f128b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:44.365586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861e9ed-b20d-4d41-81e2-66c274f128b8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:44.365862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861e9ed-b20d-4d41-81e2-66c274f128b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:44.366318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861e9ed-b20d-4d41-81e2-66c274f128b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:44.368583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861e9ed-b20d-4d41-81e2-66c274f128b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:44.368890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861e9ed-b20d-4d41-81e2-66c274f128b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:44.370691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861e9ed-b20d-4d41-81e2-66c274f128b8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:44.382252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 551/1102] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:44.389022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861e9ed-b20d-4d41-81e2-66c274f128b8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:44.420102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b44017-30b0-4204-9de5-26e61bdc65c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:44.421480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b44017-30b0-4204-9de5-26e61bdc65c4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:44.484267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861e9ed-b20d-4d41-81e2-66c274f128b8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:44.486768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:44.518677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b86f1-958a-413f-892c-83f469de975f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:44.519175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 552/1103] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 05:15:44.519310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b86f1-958a-413f-892c-83f469de975f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:44.519536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b86f1-958a-413f-892c-83f469de975f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:44.519806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b86f1-958a-413f-892c-83f469de975f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:44.520518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b86f1-958a-413f-892c-83f469de975f None admin] RBAC: Adding query filter params (role.id=688834a9cd994fa08909c0296f33b5a0, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=d9168d8bbf8b43299a7748c9faa099b1) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:44.520825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b86f1-958a-413f-892c-83f469de975f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:44.522267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b86f1-958a-413f-892c-83f469de975f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:44.524928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b44017-30b0-4204-9de5-26e61bdc65c4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:44.527321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:44.528620 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 552/1104] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:44 2022] GET /identity/v3/role_assignments?user.id=d9168d8bbf8b43299a7748c9faa099b1&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=688834a9cd994fa08909c0296f33b5a0 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:44.633035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b44017-30b0-4204-9de5-26e61bdc65c4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:44.669124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277d040-6af2-4a14-8221-e077d7899378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:44.669773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277d040-6af2-4a14-8221-e077d7899378 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:44.670083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277d040-6af2-4a14-8221-e077d7899378 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:44.670411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277d040-6af2-4a14-8221-e077d7899378 None admin] PATH_INFO: `/v3/projects/97051a34541c485eba1b2aab1fd0eb5d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:44.671514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277d040-6af2-4a14-8221-e077d7899378 None admin] RBAC: Authorizing `identity:get_project(project_id=97051a34541c485eba1b2aab1fd0eb5d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:44.672959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277d040-6af2-4a14-8221-e077d7899378 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:44.674660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 553/1105] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:15:44 2022] GET /identity/v3/projects/97051a34541c485eba1b2aab1fd0eb5d => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:44.680950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b86f1-958a-413f-892c-83f469de975f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:44.783206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b86f1-958a-413f-892c-83f469de975f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:44.819109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397a4d5-6476-4bcd-a5e9-580b00057d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:44.819723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397a4d5-6476-4bcd-a5e9-580b00057d70 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:44.820007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397a4d5-6476-4bcd-a5e9-580b00057d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:44.820285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397a4d5-6476-4bcd-a5e9-580b00057d70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:44.821009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397a4d5-6476-4bcd-a5e9-580b00057d70 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=97051a34541c485eba1b2aab1fd0eb5d, user.id=7e7c384e4153465b986697afba525773) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:44.821320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397a4d5-6476-4bcd-a5e9-580b00057d70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:44.823408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397a4d5-6476-4bcd-a5e9-580b00057d70 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:44.829704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 553/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:44 2022] GET /identity/v3/role_assignments?user.id=7e7c384e4153465b986697afba525773&scope.project.id=97051a34541c485eba1b2aab1fd0eb5d&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:15:46.986582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc20058-9cf5-4689-aad3-59bdf2bb0057 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:46.987079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc20058-9cf5-4689-aad3-59bdf2bb0057 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:46.987079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc20058-9cf5-4689-aad3-59bdf2bb0057 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:46.988641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 554/1107] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:46.993713 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985b94e-0959-461e-8bc2-f319540dc4a6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:46.994591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985b94e-0959-461e-8bc2-f319540dc4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:46.994824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985b94e-0959-461e-8bc2-f319540dc4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:47.014115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e985b94e-0959-461e-8bc2-f319540dc4a6 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:47.015283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e985b94e-0959-461e-8bc2-f319540dc4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:47.039240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:47.078657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 554/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:47.345488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc20058-9cf5-4689-aad3-59bdf2bb0057 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:47.413701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:47.445305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc20058-9cf5-4689-aad3-59bdf2bb0057 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:47.477126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7caf7-c096-4fe2-a043-50e84b1d1892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:47.477734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7caf7-c096-4fe2-a043-50e84b1d1892 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:47.478035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7caf7-c096-4fe2-a043-50e84b1d1892 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:47.478309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7caf7-c096-4fe2-a043-50e84b1d1892 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:47.479909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7caf7-c096-4fe2-a043-50e84b1d1892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:47.481321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7caf7-c096-4fe2-a043-50e84b1d1892 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:47.482882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 555/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:47.488738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985b94e-0959-461e-8bc2-f319540dc4a6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:47.582110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985b94e-0959-461e-8bc2-f319540dc4a6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:47.615726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5deb38-23c7-4ded-8e6b-c4d308fa3d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:47.616261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5deb38-23c7-4ded-8e6b-c4d308fa3d1a None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:47.616535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5deb38-23c7-4ded-8e6b-c4d308fa3d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:47.616800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5deb38-23c7-4ded-8e6b-c4d308fa3d1a None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:47.617442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5deb38-23c7-4ded-8e6b-c4d308fa3d1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:47.618880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5deb38-23c7-4ded-8e6b-c4d308fa3d1a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:47.639890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 555/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:15:47 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:15:49.794459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012270f-2c35-45b5-9e05-6bf4728f12e1 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:49.795002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012270f-2c35-45b5-9e05-6bf4728f12e1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:49.795002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012270f-2c35-45b5-9e05-6bf4728f12e1 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:49.796315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 556/1111] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:49.801582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9cf6f-7ec4-487d-8d39-787fd9473237 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:49.802005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9cf6f-7ec4-487d-8d39-787fd9473237 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:49.802166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9cf6f-7ec4-487d-8d39-787fd9473237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:49.819876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c9c9cf6f-7ec4-487d-8d39-787fd9473237 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:49.821063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c9c9cf6f-7ec4-487d-8d39-787fd9473237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:49.834435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:49.887819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 556/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:50.153091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012270f-2c35-45b5-9e05-6bf4728f12e1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:50.256646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012270f-2c35-45b5-9e05-6bf4728f12e1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:50.289269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b42e3-eff1-4954-9768-e6a2f8575097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:50.289888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b42e3-eff1-4954-9768-e6a2f8575097 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:50.290197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b42e3-eff1-4954-9768-e6a2f8575097 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:50.290468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b42e3-eff1-4954-9768-e6a2f8575097 None admin] PATH_INFO: `/v3/roles/9578cd8bc0a8498f99757fd0111e48e8` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:50.291838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b42e3-eff1-4954-9768-e6a2f8575097 None admin] RBAC: Authorizing `identity:get_role(role_id=9578cd8bc0a8498f99757fd0111e48e8)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:50.293197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b42e3-eff1-4954-9768-e6a2f8575097 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:50.294573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 557/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:15:50 2022] GET /identity/v3/roles/9578cd8bc0a8498f99757fd0111e48e8 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:50.300420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9cf6f-7ec4-487d-8d39-787fd9473237 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:50.395483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9cf6f-7ec4-487d-8d39-787fd9473237 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:50.428622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c5bee-9044-439b-aca7-8ff550e6aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:50.429177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c5bee-9044-439b-aca7-8ff550e6aa7f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:50.429454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c5bee-9044-439b-aca7-8ff550e6aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:50.429727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c5bee-9044-439b-aca7-8ff550e6aa7f None admin] PATH_INFO: `/v3/users/fc685ca391a34404966e66ce83cd6290` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:50.443128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c5bee-9044-439b-aca7-8ff550e6aa7f None admin] RBAC: Authorizing `identity:get_user(user_id=fc685ca391a34404966e66ce83cd6290)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:50.444542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c5bee-9044-439b-aca7-8ff550e6aa7f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:50.446136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 557/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:15:50 2022] GET /identity/v3/users/fc685ca391a34404966e66ce83cd6290 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:50.451870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b42e3-eff1-4954-9768-e6a2f8575097 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:50.460491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:50.549927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b42e3-eff1-4954-9768-e6a2f8575097 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:50.582480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd911b99-861a-485c-8e4a-e9d04b032a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:50.583069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd911b99-861a-485c-8e4a-e9d04b032a3d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:50.583346 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd911b99-861a-485c-8e4a-e9d04b032a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:50.583622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd911b99-861a-485c-8e4a-e9d04b032a3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:50.649396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd911b99-861a-485c-8e4a-e9d04b032a3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:50.650912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd911b99-861a-485c-8e4a-e9d04b032a3d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:50.658144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:50.751493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 558/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:15:50.757375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c5bee-9044-439b-aca7-8ff550e6aa7f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:50.766074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:50.852746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c5bee-9044-439b-aca7-8ff550e6aa7f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:50.885371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b346a-76e1-4f96-a4ee-5737acfcf69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:50.886010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b346a-76e1-4f96-a4ee-5737acfcf69f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:50.886289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b346a-76e1-4f96-a4ee-5737acfcf69f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:50.886560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b346a-76e1-4f96-a4ee-5737acfcf69f None admin] PATH_INFO: `/v3/projects/97051a34541c485eba1b2aab1fd0eb5d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:50.887646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b346a-76e1-4f96-a4ee-5737acfcf69f None admin] RBAC: Authorizing `identity:get_project(project_id=97051a34541c485eba1b2aab1fd0eb5d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:50.889049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b346a-76e1-4f96-a4ee-5737acfcf69f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:50.890667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 558/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:15:50 2022] GET /identity/v3/projects/97051a34541c485eba1b2aab1fd0eb5d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:50.896806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd911b99-861a-485c-8e4a-e9d04b032a3d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:50.992293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd911b99-861a-485c-8e4a-e9d04b032a3d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:50.999236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:51.025881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9afa30-321e-41ac-bd6c-b987f9f36dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:51.026444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9afa30-321e-41ac-bd6c-b987f9f36dbe None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:51.026724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9afa30-321e-41ac-bd6c-b987f9f36dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:51.026997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9afa30-321e-41ac-bd6c-b987f9f36dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:51.027712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9afa30-321e-41ac-bd6c-b987f9f36dbe None admin] RBAC: Adding query filter params (role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=97051a34541c485eba1b2aab1fd0eb5d, user.id=fc685ca391a34404966e66ce83cd6290) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:51.028023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9afa30-321e-41ac-bd6c-b987f9f36dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:51.029441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9afa30-321e-41ac-bd6c-b987f9f36dbe None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:51.034629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 559/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:50 2022] GET /identity/v3/role_assignments?user.id=fc685ca391a34404966e66ce83cd6290&scope.project.id=97051a34541c485eba1b2aab1fd0eb5d&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:53.159000 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592fad9-453b-46bf-ae20-9e030fd5c645 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:53.159516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592fad9-453b-46bf-ae20-9e030fd5c645 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:53.159607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592fad9-453b-46bf-ae20-9e030fd5c645 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:53.160849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 559/1118] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:53.166109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60405565-54e1-445f-b068-953de8499942 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:53.166499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60405565-54e1-445f-b068-953de8499942 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:53.166662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60405565-54e1-445f-b068-953de8499942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:53.184719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-60405565-54e1-445f-b068-953de8499942 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:53.185899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-60405565-54e1-445f-b068-953de8499942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:53.249423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 560/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:53.519146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592fad9-453b-46bf-ae20-9e030fd5c645 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:53.593115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:53.618831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592fad9-453b-46bf-ae20-9e030fd5c645 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:53.651926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fad77-0237-4cf9-9c46-b2bf22fb56cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:53.652544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fad77-0237-4cf9-9c46-b2bf22fb56cc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:53.652830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fad77-0237-4cf9-9c46-b2bf22fb56cc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:53.653099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fad77-0237-4cf9-9c46-b2bf22fb56cc None admin] PATH_INFO: `/v3/roles/9578cd8bc0a8498f99757fd0111e48e8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:53.654496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fad77-0237-4cf9-9c46-b2bf22fb56cc None admin] RBAC: Authorizing `identity:get_role(role_id=9578cd8bc0a8498f99757fd0111e48e8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:53.655837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fad77-0237-4cf9-9c46-b2bf22fb56cc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:53.657119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 560/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:15:53 2022] GET /identity/v3/roles/9578cd8bc0a8498f99757fd0111e48e8 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:53.662966 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60405565-54e1-445f-b068-953de8499942 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:53.756621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:53.759641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60405565-54e1-445f-b068-953de8499942 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:53.793713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148080dd-ac83-42e1-966f-ea7b9cd97a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:53.794301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148080dd-ac83-42e1-966f-ea7b9cd97a7a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:53.794581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148080dd-ac83-42e1-966f-ea7b9cd97a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:53.794852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148080dd-ac83-42e1-966f-ea7b9cd97a7a None admin] PATH_INFO: `/v3/users/fc685ca391a34404966e66ce83cd6290` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:53.795953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148080dd-ac83-42e1-966f-ea7b9cd97a7a None admin] RBAC: Authorizing `identity:get_user(user_id=fc685ca391a34404966e66ce83cd6290)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:53.797381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148080dd-ac83-42e1-966f-ea7b9cd97a7a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:53.798964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 561/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:15:53 2022] GET /identity/v3/users/fc685ca391a34404966e66ce83cd6290 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:53.804730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fad77-0237-4cf9-9c46-b2bf22fb56cc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:53.899521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fad77-0237-4cf9-9c46-b2bf22fb56cc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:53.932337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb06f57-5768-4aec-9dc1-a03cdf19f92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:53.932945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06f57-5768-4aec-9dc1-a03cdf19f92e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:53.933223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06f57-5768-4aec-9dc1-a03cdf19f92e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:53.933496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06f57-5768-4aec-9dc1-a03cdf19f92e None admin] PATH_INFO: `/v3/projects/97051a34541c485eba1b2aab1fd0eb5d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:53.934609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb06f57-5768-4aec-9dc1-a03cdf19f92e None admin] RBAC: Authorizing `identity:get_project(project_id=97051a34541c485eba1b2aab1fd0eb5d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:53.936018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb06f57-5768-4aec-9dc1-a03cdf19f92e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:53.937585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 561/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:15:53 2022] GET /identity/v3/projects/97051a34541c485eba1b2aab1fd0eb5d => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:53.943392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148080dd-ac83-42e1-966f-ea7b9cd97a7a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:54.039114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148080dd-ac83-42e1-966f-ea7b9cd97a7a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:54.072504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379e963-547e-4d81-89f5-a68be41a6536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:54.073115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379e963-547e-4d81-89f5-a68be41a6536 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:54.073396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379e963-547e-4d81-89f5-a68be41a6536 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:54.073672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379e963-547e-4d81-89f5-a68be41a6536 None admin] PATH_INFO: `/v3/projects/97051a34541c485eba1b2aab1fd0eb5d/users/fc685ca391a34404966e66ce83cd6290/roles/9578cd8bc0a8498f99757fd0111e48e8` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:54.075802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379e963-547e-4d81-89f5-a68be41a6536 None admin] RBAC: Authorizing `identity:create_grant(project_id=97051a34541c485eba1b2aab1fd0eb5d, user_id=fc685ca391a34404966e66ce83cd6290, role_id=9578cd8bc0a8498f99757fd0111e48e8)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:54.077216 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379e963-547e-4d81-89f5-a68be41a6536 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:54.089999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 562/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:15:53 2022] PUT /identity/v3/projects/97051a34541c485eba1b2aab1fd0eb5d/users/fc685ca391a34404966e66ce83cd6290/roles/9578cd8bc0a8498f99757fd0111e48e8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:15:56.174270 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b052869-fea1-42c4-a02d-e67771a00fa9 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:56.174797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b052869-fea1-42c4-a02d-e67771a00fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:56.174797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b052869-fea1-42c4-a02d-e67771a00fa9 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:56.176121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 562/1124] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:56.181227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd4160-95a8-40c6-b035-0207f4bef30e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:56.182208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd4160-95a8-40c6-b035-0207f4bef30e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:56.182324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd4160-95a8-40c6-b035-0207f4bef30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:56.200555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9ecd4160-95a8-40c6-b035-0207f4bef30e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:56.201734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9ecd4160-95a8-40c6-b035-0207f4bef30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:56.223343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:56.266877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 563/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:15:56.532480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b052869-fea1-42c4-a02d-e67771a00fa9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:56.545327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:56.631716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b052869-fea1-42c4-a02d-e67771a00fa9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:56.663068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc1d3f-2cd4-4b47-a4a8-472ecc282bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:56.663680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc1d3f-2cd4-4b47-a4a8-472ecc282bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:56.663957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc1d3f-2cd4-4b47-a4a8-472ecc282bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:56.664228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc1d3f-2cd4-4b47-a4a8-472ecc282bf0 None admin] PATH_INFO: `/v3/roles/9578cd8bc0a8498f99757fd0111e48e8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:56.665574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc1d3f-2cd4-4b47-a4a8-472ecc282bf0 None admin] RBAC: Authorizing `identity:get_role(role_id=9578cd8bc0a8498f99757fd0111e48e8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:56.667464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc1d3f-2cd4-4b47-a4a8-472ecc282bf0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:56.668739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 563/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:15:56 2022] GET /identity/v3/roles/9578cd8bc0a8498f99757fd0111e48e8 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:56.674617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd4160-95a8-40c6-b035-0207f4bef30e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:56.771434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd4160-95a8-40c6-b035-0207f4bef30e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:56.805725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b3b25-db2b-4b19-b8ea-afacbc3c08cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:56.806333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b3b25-db2b-4b19-b8ea-afacbc3c08cf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:56.806615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b3b25-db2b-4b19-b8ea-afacbc3c08cf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:56.806888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b3b25-db2b-4b19-b8ea-afacbc3c08cf None admin] PATH_INFO: `/v3/users/fc685ca391a34404966e66ce83cd6290` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:56.807983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b3b25-db2b-4b19-b8ea-afacbc3c08cf None admin] RBAC: Authorizing `identity:get_user(user_id=fc685ca391a34404966e66ce83cd6290)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:56.809408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b3b25-db2b-4b19-b8ea-afacbc3c08cf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:56.810978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 564/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:15:56 2022] GET /identity/v3/users/fc685ca391a34404966e66ce83cd6290 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:56.816728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc1d3f-2cd4-4b47-a4a8-472ecc282bf0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:56.893375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:56.914458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc1d3f-2cd4-4b47-a4a8-472ecc282bf0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:56.946546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665d39d-f34c-4729-8aab-625430c113cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:56.947180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665d39d-f34c-4729-8aab-625430c113cf None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:56.947457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665d39d-f34c-4729-8aab-625430c113cf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:56.947725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665d39d-f34c-4729-8aab-625430c113cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:57.011118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665d39d-f34c-4729-8aab-625430c113cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:57.012522 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665d39d-f34c-4729-8aab-625430c113cf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:57.085478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:57.107385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 564/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:15:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:15:57.113515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b3b25-db2b-4b19-b8ea-afacbc3c08cf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:57.139856 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:57.209287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b3b25-db2b-4b19-b8ea-afacbc3c08cf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:57.241847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182094f-e2c9-4f70-802d-cfd0e384d10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:57.242479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182094f-e2c9-4f70-802d-cfd0e384d10b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:57.242759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182094f-e2c9-4f70-802d-cfd0e384d10b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:57.243028 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182094f-e2c9-4f70-802d-cfd0e384d10b None admin] PATH_INFO: `/v3/projects/97051a34541c485eba1b2aab1fd0eb5d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:57.244144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182094f-e2c9-4f70-802d-cfd0e384d10b None admin] RBAC: Authorizing `identity:get_project(project_id=97051a34541c485eba1b2aab1fd0eb5d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:57.245560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182094f-e2c9-4f70-802d-cfd0e384d10b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:57.247181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 565/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:15:57 2022] GET /identity/v3/projects/97051a34541c485eba1b2aab1fd0eb5d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:57.253150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665d39d-f34c-4729-8aab-625430c113cf None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:57.348383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665d39d-f34c-4729-8aab-625430c113cf None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:15:57.380665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb74d1-3dfe-4732-b7b7-c735aef6546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:15:57.381252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb74d1-3dfe-4732-b7b7-c735aef6546c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:57.381529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb74d1-3dfe-4732-b7b7-c735aef6546c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:57.381801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb74d1-3dfe-4732-b7b7-c735aef6546c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:57.382536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb74d1-3dfe-4732-b7b7-c735aef6546c None admin] RBAC: Adding query filter params (role.id=9578cd8bc0a8498f99757fd0111e48e8, scope.project.id=97051a34541c485eba1b2aab1fd0eb5d, user.id=fc685ca391a34404966e66ce83cd6290) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:15:57.382845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb74d1-3dfe-4732-b7b7-c735aef6546c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:15:57.384238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb74d1-3dfe-4732-b7b7-c735aef6546c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:15:57.390364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 565/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:15:57 2022] GET /identity/v3/role_assignments?user.id=fc685ca391a34404966e66ce83cd6290&scope.project.id=97051a34541c485eba1b2aab1fd0eb5d&role.id=9578cd8bc0a8498f99757fd0111e48e8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:15:59.522692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526f5cc-afda-44b8-8a40-de2cc12bfce8 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:59.523242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526f5cc-afda-44b8-8a40-de2cc12bfce8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:59.523242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526f5cc-afda-44b8-8a40-de2cc12bfce8 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:59.524587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 566/1131] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:15:59.529788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac76efb-381f-45f3-96f9-3b635a99d579 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:15:59.530185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac76efb-381f-45f3-96f9-3b635a99d579 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:15:59.530376 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac76efb-381f-45f3-96f9-3b635a99d579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:15:59.549479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2ac76efb-381f-45f3-96f9-3b635a99d579 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:15:59.550681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2ac76efb-381f-45f3-96f9-3b635a99d579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:15:59.584154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:59.615826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 566/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:15:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:15:59.884328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526f5cc-afda-44b8-8a40-de2cc12bfce8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:15:59.974659 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:15:59.982913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526f5cc-afda-44b8-8a40-de2cc12bfce8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:00.015157 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605706-cb4f-4f5b-a37e-1b416209e139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:00.015768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605706-cb4f-4f5b-a37e-1b416209e139 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:00.016045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605706-cb4f-4f5b-a37e-1b416209e139 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:00.016313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605706-cb4f-4f5b-a37e-1b416209e139 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:00.017390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d605706-cb4f-4f5b-a37e-1b416209e139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:00.018829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d605706-cb4f-4f5b-a37e-1b416209e139 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:00.020342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 567/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:15:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:00.026441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac76efb-381f-45f3-96f9-3b635a99d579 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:00.120955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac76efb-381f-45f3-96f9-3b635a99d579 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:00.154130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1090a7-6e17-45df-b252-e66c56f823a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:00.154668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1090a7-6e17-45df-b252-e66c56f823a4 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:00.154948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1090a7-6e17-45df-b252-e66c56f823a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:00.155220 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1090a7-6e17-45df-b252-e66c56f823a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:00.155861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1090a7-6e17-45df-b252-e66c56f823a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:00.157283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1090a7-6e17-45df-b252-e66c56f823a4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:00.182400 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 567/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:16:00 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:16:02.318542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bee4e0-c56f-4715-9889-9bd5bc845b9d None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:02.319053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bee4e0-c56f-4715-9889-9bd5bc845b9d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:02.319053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bee4e0-c56f-4715-9889-9bd5bc845b9d None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:02.320399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 568/1135] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:02.326438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1ec88-5322-4f83-9c5c-7840ebfc0bdc None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:02.326803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1ec88-5322-4f83-9c5c-7840ebfc0bdc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:02.326994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1ec88-5322-4f83-9c5c-7840ebfc0bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:02.344735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-66d1ec88-5322-4f83-9c5c-7840ebfc0bdc None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:02.345916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-66d1ec88-5322-4f83-9c5c-7840ebfc0bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:02.363344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:02.411446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 568/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:02.680506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bee4e0-c56f-4715-9889-9bd5bc845b9d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:02.782990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bee4e0-c56f-4715-9889-9bd5bc845b9d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:02.815081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd93fa-804d-4aaf-bdde-8564962da6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:02.815685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd93fa-804d-4aaf-bdde-8564962da6bd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:02.815961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd93fa-804d-4aaf-bdde-8564962da6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:02.816231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd93fa-804d-4aaf-bdde-8564962da6bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:02.817304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd93fa-804d-4aaf-bdde-8564962da6bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:02.818743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd93fa-804d-4aaf-bdde-8564962da6bd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:02.820254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 569/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:16:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:02.825883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1ec88-5322-4f83-9c5c-7840ebfc0bdc None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:02.925051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1ec88-5322-4f83-9c5c-7840ebfc0bdc None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:02.957001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b321cb8-5183-438d-9a21-bd3e85415dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:02.957601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b321cb8-5183-438d-9a21-bd3e85415dfe None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:02.957883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b321cb8-5183-438d-9a21-bd3e85415dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:02.958185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b321cb8-5183-438d-9a21-bd3e85415dfe None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:02.958837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b321cb8-5183-438d-9a21-bd3e85415dfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:02.960250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b321cb8-5183-438d-9a21-bd3e85415dfe None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:02.982130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 569/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:16:02 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:16:05.135405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8b4dd-49dd-48ad-9dd5-8576cc0bcfb9 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:05.135935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8b4dd-49dd-48ad-9dd5-8576cc0bcfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:05.135935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8b4dd-49dd-48ad-9dd5-8576cc0bcfb9 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:05.137258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 570/1139] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:05.142506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf2d4e-a8fc-400e-9148-6aace8942aea None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:05.142906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf2d4e-a8fc-400e-9148-6aace8942aea None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:05.143061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf2d4e-a8fc-400e-9148-6aace8942aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:05.160836 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0cbf2d4e-a8fc-400e-9148-6aace8942aea None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:05.162059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0cbf2d4e-a8fc-400e-9148-6aace8942aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:05.164720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:05.227384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 570/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:05.495268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8b4dd-49dd-48ad-9dd5-8576cc0bcfb9 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:05.524532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:05.595795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8b4dd-49dd-48ad-9dd5-8576cc0bcfb9 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:05.629665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4db7e9-51af-4319-8fc3-ce54c3e7f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:05.630316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4db7e9-51af-4319-8fc3-ce54c3e7f170 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:05.630597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4db7e9-51af-4319-8fc3-ce54c3e7f170 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:05.630870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4db7e9-51af-4319-8fc3-ce54c3e7f170 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:05.641194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4db7e9-51af-4319-8fc3-ce54c3e7f170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5b4db7e9-51af-4319-8fc3-ce54c3e7f170 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:16:05.645949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:16:05.647048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:16:05.648240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 571/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:16:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:16:05.653647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf2d4e-a8fc-400e-9148-6aace8942aea None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:05.747834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf2d4e-a8fc-400e-9148-6aace8942aea None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:05.779850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2c7ad-ee1a-440a-881d-1b143f36ea52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:05.780434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2c7ad-ee1a-440a-881d-1b143f36ea52 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:05.780709 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2c7ad-ee1a-440a-881d-1b143f36ea52 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:05.780975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2c7ad-ee1a-440a-881d-1b143f36ea52 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:05.781608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2c7ad-ee1a-440a-881d-1b143f36ea52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:05.781905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2c7ad-ee1a-440a-881d-1b143f36ea52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:05.783265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2c7ad-ee1a-440a-881d-1b143f36ea52 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:05.790990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 571/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:16:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:05.796688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4db7e9-51af-4319-8fc3-ce54c3e7f170 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:05.888180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:05.898188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4db7e9-51af-4319-8fc3-ce54c3e7f170 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:05.931569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e0b95-6975-4d39-ac86-8700879136a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:05.932186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e0b95-6975-4d39-ac86-8700879136a1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:05.932466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e0b95-6975-4d39-ac86-8700879136a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:05.932741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e0b95-6975-4d39-ac86-8700879136a1 None admin] PATH_INFO: `/v3/users/8e94ec6f0126470f8859ea54530a0b1b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:05.945826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e0b95-6975-4d39-ac86-8700879136a1 None admin] RBAC: Authorizing `identity:get_user(user_id=8e94ec6f0126470f8859ea54530a0b1b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:05.947910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e0b95-6975-4d39-ac86-8700879136a1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:05.949520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 572/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:16:05 2022] GET /identity/v3/users/8e94ec6f0126470f8859ea54530a0b1b => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:05.955512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2c7ad-ee1a-440a-881d-1b143f36ea52 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:05.958368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:06.053138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2c7ad-ee1a-440a-881d-1b143f36ea52 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:06.085853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da842f-5615-464a-b22e-1ac72d35156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:06.086482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da842f-5615-464a-b22e-1ac72d35156a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:06.086758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da842f-5615-464a-b22e-1ac72d35156a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:06.087028 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da842f-5615-464a-b22e-1ac72d35156a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:06.151706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da842f-5615-464a-b22e-1ac72d35156a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:06.153126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da842f-5615-464a-b22e-1ac72d35156a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:06.154858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:06.248603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 572/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:16:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:16:06.254696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e0b95-6975-4d39-ac86-8700879136a1 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:06.349877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e0b95-6975-4d39-ac86-8700879136a1 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:06.381974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa216c-78b5-434d-abd0-b4345268cdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:06.382584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa216c-78b5-434d-abd0-b4345268cdbd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:06.382859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa216c-78b5-434d-abd0-b4345268cdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:06.383128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa216c-78b5-434d-abd0-b4345268cdbd None admin] PATH_INFO: `/v3/projects/ee1cc96a10b54acbbba62243a95a922b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:06.384713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa216c-78b5-434d-abd0-b4345268cdbd None admin] RBAC: Authorizing `identity:get_project(project_id=ee1cc96a10b54acbbba62243a95a922b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:06.386140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa216c-78b5-434d-abd0-b4345268cdbd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:06.387720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 573/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:16:06 2022] GET /identity/v3/projects/ee1cc96a10b54acbbba62243a95a922b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:06.393753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da842f-5615-464a-b22e-1ac72d35156a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:06.488658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da842f-5615-464a-b22e-1ac72d35156a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:06.490146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:06.522468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552f653-5679-4ad3-b143-d600a626e8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:06.523057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f653-5679-4ad3-b143-d600a626e8dc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:06.523359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f653-5679-4ad3-b143-d600a626e8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:06.523631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f653-5679-4ad3-b143-d600a626e8dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:06.524338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f653-5679-4ad3-b143-d600a626e8dc None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=ee1cc96a10b54acbbba62243a95a922b, user.id=8e94ec6f0126470f8859ea54530a0b1b) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:06.524643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f653-5679-4ad3-b143-d600a626e8dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:06.526060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f653-5679-4ad3-b143-d600a626e8dc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:06.531207 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 573/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:16:06 2022] GET /identity/v3/role_assignments?user.id=8e94ec6f0126470f8859ea54530a0b1b&scope.project.id=ee1cc96a10b54acbbba62243a95a922b&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:08.662522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d12ed3-1f95-47f8-b5c6-0fca7e7687ae None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:08.663023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d12ed3-1f95-47f8-b5c6-0fca7e7687ae None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:08.663107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d12ed3-1f95-47f8-b5c6-0fca7e7687ae None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:08.664399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 574/1147] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:08.669692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9b6b-4b17-4892-9cbd-429eb03bf2fe None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:08.670147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9b6b-4b17-4892-9cbd-429eb03bf2fe None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:08.670264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9b6b-4b17-4892-9cbd-429eb03bf2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:08.689150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-83bb9b6b-4b17-4892-9cbd-429eb03bf2fe None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:08.690660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-83bb9b6b-4b17-4892-9cbd-429eb03bf2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:08.760474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 574/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:09.035989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d12ed3-1f95-47f8-b5c6-0fca7e7687ae None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:09.054269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:09.138745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d12ed3-1f95-47f8-b5c6-0fca7e7687ae None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:09.171075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcf85d-1b2e-42e0-a39e-de50c6b03851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:09.171691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcf85d-1b2e-42e0-a39e-de50c6b03851 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:09.171976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcf85d-1b2e-42e0-a39e-de50c6b03851 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:09.172249 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcf85d-1b2e-42e0-a39e-de50c6b03851 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:09.181981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcf85d-1b2e-42e0-a39e-de50c6b03851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-9adcf85d-1b2e-42e0-a39e-de50c6b03851 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:16:09.186667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:16:09.187779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:16:09.188831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 575/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:16:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:16:09.193864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb9b6b-4b17-4892-9cbd-429eb03bf2fe None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:09.283887 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:09.290018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb9b6b-4b17-4892-9cbd-429eb03bf2fe None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:09.322524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853b5cd-bd64-46b8-aa77-bc99a146d95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:09.323113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853b5cd-bd64-46b8-aa77-bc99a146d95c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:09.323390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853b5cd-bd64-46b8-aa77-bc99a146d95c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:09.323657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853b5cd-bd64-46b8-aa77-bc99a146d95c None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:09.324298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853b5cd-bd64-46b8-aa77-bc99a146d95c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:09.324593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853b5cd-bd64-46b8-aa77-bc99a146d95c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:09.325961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853b5cd-bd64-46b8-aa77-bc99a146d95c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:09.333559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 575/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:16:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:09.339907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcf85d-1b2e-42e0-a39e-de50c6b03851 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:09.415130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:09.437635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcf85d-1b2e-42e0-a39e-de50c6b03851 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:09.471063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ff20d-4342-463e-90b4-cf5f20e33a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:09.471679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ff20d-4342-463e-90b4-cf5f20e33a6c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:09.471957 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ff20d-4342-463e-90b4-cf5f20e33a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:09.472228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ff20d-4342-463e-90b4-cf5f20e33a6c None admin] PATH_INFO: `/v3/users/8e94ec6f0126470f8859ea54530a0b1b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:09.473324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ff20d-4342-463e-90b4-cf5f20e33a6c None admin] RBAC: Authorizing `identity:get_user(user_id=8e94ec6f0126470f8859ea54530a0b1b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:09.474776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ff20d-4342-463e-90b4-cf5f20e33a6c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:09.476332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 576/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:16:09 2022] GET /identity/v3/users/8e94ec6f0126470f8859ea54530a0b1b => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:09.482056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853b5cd-bd64-46b8-aa77-bc99a146d95c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:09.579341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853b5cd-bd64-46b8-aa77-bc99a146d95c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:09.612123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c901cd2-3dea-4af6-b8ff-e635fe819065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:09.613004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c901cd2-3dea-4af6-b8ff-e635fe819065 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:09.613281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c901cd2-3dea-4af6-b8ff-e635fe819065 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:09.613549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c901cd2-3dea-4af6-b8ff-e635fe819065 None admin] PATH_INFO: `/v3/projects/ee1cc96a10b54acbbba62243a95a922b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:09.614671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c901cd2-3dea-4af6-b8ff-e635fe819065 None admin] RBAC: Authorizing `identity:get_project(project_id=ee1cc96a10b54acbbba62243a95a922b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:09.616069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c901cd2-3dea-4af6-b8ff-e635fe819065 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:09.617659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 576/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:16:09 2022] GET /identity/v3/projects/ee1cc96a10b54acbbba62243a95a922b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:09.624087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ff20d-4342-463e-90b4-cf5f20e33a6c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:09.719359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ff20d-4342-463e-90b4-cf5f20e33a6c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:09.752433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadee39-c346-4373-8b37-4eb0610b876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:09.753074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadee39-c346-4373-8b37-4eb0610b876b None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:09.753352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadee39-c346-4373-8b37-4eb0610b876b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:09.753622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadee39-c346-4373-8b37-4eb0610b876b None admin] PATH_INFO: `/v3/projects/ee1cc96a10b54acbbba62243a95a922b/users/8e94ec6f0126470f8859ea54530a0b1b/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:09.755498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadee39-c346-4373-8b37-4eb0610b876b None admin] RBAC: Authorizing `identity:create_grant(project_id=ee1cc96a10b54acbbba62243a95a922b, user_id=8e94ec6f0126470f8859ea54530a0b1b, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:09.756924 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadee39-c346-4373-8b37-4eb0610b876b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:09.771306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 577/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:16:09 2022] PUT /identity/v3/projects/ee1cc96a10b54acbbba62243a95a922b/users/8e94ec6f0126470f8859ea54530a0b1b/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:16:11.881635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589d290-6fde-48a3-b807-0f9ae639cc58 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:11.882176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589d290-6fde-48a3-b807-0f9ae639cc58 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:11.882176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589d290-6fde-48a3-b807-0f9ae639cc58 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:11.883455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 577/1154] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:11.888652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56861-b5ac-45a4-af28-5d4890b3ebac None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:11.889027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56861-b5ac-45a4-af28-5d4890b3ebac None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:11.889197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56861-b5ac-45a4-af28-5d4890b3ebac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:11.896349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:11.908271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-93b56861-b5ac-45a4-af28-5d4890b3ebac None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:11.909440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-93b56861-b5ac-45a4-af28-5d4890b3ebac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:11.972964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 578/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:12.240761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589d290-6fde-48a3-b807-0f9ae639cc58 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:12.267904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:12.340670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589d290-6fde-48a3-b807-0f9ae639cc58 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:12.373622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a5ac9-0a89-4ed6-ba01-e476c8199215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:12.374266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a5ac9-0a89-4ed6-ba01-e476c8199215 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:12.374545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a5ac9-0a89-4ed6-ba01-e476c8199215 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:12.374814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a5ac9-0a89-4ed6-ba01-e476c8199215 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:12.384482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a5ac9-0a89-4ed6-ba01-e476c8199215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4d1a5ac9-0a89-4ed6-ba01-e476c8199215 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:16:12.389197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:16:12.390402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:16:12.391421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 578/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:16:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:16:12.396276 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56861-b5ac-45a4-af28-5d4890b3ebac None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:12.492657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56861-b5ac-45a4-af28-5d4890b3ebac None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:12.525170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebbc0f-5e3b-4b86-8cff-f4c9655306ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:12.525759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbc0f-5e3b-4b86-8cff-f4c9655306ae None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:12.526074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbc0f-5e3b-4b86-8cff-f4c9655306ae None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:12.526346 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbc0f-5e3b-4b86-8cff-f4c9655306ae None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:12.526989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebbc0f-5e3b-4b86-8cff-f4c9655306ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:12.527289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebbc0f-5e3b-4b86-8cff-f4c9655306ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:12.528644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebbc0f-5e3b-4b86-8cff-f4c9655306ae None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:12.529917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:12.537297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 579/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:16:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:12.543843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a5ac9-0a89-4ed6-ba01-e476c8199215 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:12.626543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:12.640605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a5ac9-0a89-4ed6-ba01-e476c8199215 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:12.673695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ae575-b733-4788-880a-237cd0ef35f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:12.674385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ae575-b733-4788-880a-237cd0ef35f6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:12.674666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ae575-b733-4788-880a-237cd0ef35f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:12.674938 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ae575-b733-4788-880a-237cd0ef35f6 None admin] PATH_INFO: `/v3/users/8e94ec6f0126470f8859ea54530a0b1b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:12.676038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ae575-b733-4788-880a-237cd0ef35f6 None admin] RBAC: Authorizing `identity:get_user(user_id=8e94ec6f0126470f8859ea54530a0b1b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:12.677486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ae575-b733-4788-880a-237cd0ef35f6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:12.679072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 579/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:16:12 2022] GET /identity/v3/users/8e94ec6f0126470f8859ea54530a0b1b => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:12.684847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebbc0f-5e3b-4b86-8cff-f4c9655306ae None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:12.781173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebbc0f-5e3b-4b86-8cff-f4c9655306ae None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:12.813355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166714b1-30a4-463f-9e80-5eef465f0bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:12.813964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166714b1-30a4-463f-9e80-5eef465f0bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:12.814273 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166714b1-30a4-463f-9e80-5eef465f0bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:12.814544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166714b1-30a4-463f-9e80-5eef465f0bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:12.873206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:12.879986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166714b1-30a4-463f-9e80-5eef465f0bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:12.881415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166714b1-30a4-463f-9e80-5eef465f0bc7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:12.977367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 580/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:16:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:16:12.983743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ae575-b733-4788-880a-237cd0ef35f6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:13.079261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ae575-b733-4788-880a-237cd0ef35f6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:13.111287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90adcc-6941-4f02-8319-436837870071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:13.111902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90adcc-6941-4f02-8319-436837870071 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:13.112178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90adcc-6941-4f02-8319-436837870071 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:13.112705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90adcc-6941-4f02-8319-436837870071 None admin] PATH_INFO: `/v3/projects/ee1cc96a10b54acbbba62243a95a922b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:13.113782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90adcc-6941-4f02-8319-436837870071 None admin] RBAC: Authorizing `identity:get_project(project_id=ee1cc96a10b54acbbba62243a95a922b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:13.115205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90adcc-6941-4f02-8319-436837870071 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:13.116779 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 580/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:16:12 2022] GET /identity/v3/projects/ee1cc96a10b54acbbba62243a95a922b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:13.123355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166714b1-30a4-463f-9e80-5eef465f0bc7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:13.212658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:13.218827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166714b1-30a4-463f-9e80-5eef465f0bc7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:13.258210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3022e73-98e7-4192-be27-4e8a1fe222ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:13.258797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3022e73-98e7-4192-be27-4e8a1fe222ff None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:13.259095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3022e73-98e7-4192-be27-4e8a1fe222ff None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:13.259365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3022e73-98e7-4192-be27-4e8a1fe222ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:13.260076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3022e73-98e7-4192-be27-4e8a1fe222ff None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=ee1cc96a10b54acbbba62243a95a922b, user.id=8e94ec6f0126470f8859ea54530a0b1b) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:13.260380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3022e73-98e7-4192-be27-4e8a1fe222ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:13.261793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3022e73-98e7-4192-be27-4e8a1fe222ff None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:13.267438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 581/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:16:13 2022] GET /identity/v3/role_assignments?user.id=8e94ec6f0126470f8859ea54530a0b1b&scope.project.id=ee1cc96a10b54acbbba62243a95a922b&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:15.414885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80102b-e4a6-41d4-a600-30589765a048 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:15.415952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80102b-e4a6-41d4-a600-30589765a048 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:15.415952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80102b-e4a6-41d4-a600-30589765a048 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:15.417288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 581/1162] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:15.422640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e353-6a3b-408a-983f-9a8443f7a148 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:15.423003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e353-6a3b-408a-983f-9a8443f7a148 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:15.423182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e353-6a3b-408a-983f-9a8443f7a148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:15.441205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-daa4e353-6a3b-408a-983f-9a8443f7a148 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:15.442422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-daa4e353-6a3b-408a-983f-9a8443f7a148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:15.506740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 582/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:15.771544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80102b-e4a6-41d4-a600-30589765a048 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:15.796254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:15.870884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80102b-e4a6-41d4-a600-30589765a048 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:15.903989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59182e-f92d-46a0-a8f0-422914f0a6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:15.904582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59182e-f92d-46a0-a8f0-422914f0a6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:15.904858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59182e-f92d-46a0-a8f0-422914f0a6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:15.905129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59182e-f92d-46a0-a8f0-422914f0a6f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:15.968784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59182e-f92d-46a0-a8f0-422914f0a6f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:15.970227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59182e-f92d-46a0-a8f0-422914f0a6f0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:15.992616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:16.073866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 582/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:16:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:16:16.079755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4e353-6a3b-408a-983f-9a8443f7a148 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:16.165540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:16.175020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4e353-6a3b-408a-983f-9a8443f7a148 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:16.208137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f87b0-746a-482e-8221-0c77777c4f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:16.208750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f87b0-746a-482e-8221-0c77777c4f4b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:16.209028 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f87b0-746a-482e-8221-0c77777c4f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:16.209299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f87b0-746a-482e-8221-0c77777c4f4b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:16.216296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f87b0-746a-482e-8221-0c77777c4f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:16.217723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f87b0-746a-482e-8221-0c77777c4f4b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5f3f87b0-746a-482e-8221-0c77777c4f4b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:16:16.228765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:16:16.229962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:16:16.230997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 583/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:16:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:16:16.235872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59182e-f92d-46a0-a8f0-422914f0a6f0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:16.331436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59182e-f92d-46a0-a8f0-422914f0a6f0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:16.352770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:16.364527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85675b41-fd02-47dd-ac0b-ae309551d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:16.365123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85675b41-fd02-47dd-ac0b-ae309551d463 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:16.365401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85675b41-fd02-47dd-ac0b-ae309551d463 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:16.365675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85675b41-fd02-47dd-ac0b-ae309551d463 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:16.366335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85675b41-fd02-47dd-ac0b-ae309551d463 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:16.366638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85675b41-fd02-47dd-ac0b-ae309551d463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:16.367991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85675b41-fd02-47dd-ac0b-ae309551d463 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:16.375670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 583/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Sep 21 05:16:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:18.503374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5932-8097-4ccd-802f-bc65c041fbc3 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:18.503934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5932-8097-4ccd-802f-bc65c041fbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:18.503934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5932-8097-4ccd-802f-bc65c041fbc3 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:18.505252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 584/1167] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:18.511226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f80a-cd11-471b-9a4d-ece2b8b6272f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:18.511611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f80a-cd11-471b-9a4d-ece2b8b6272f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:18.511776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f80a-cd11-471b-9a4d-ece2b8b6272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:18.529572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3269f80a-cd11-471b-9a4d-ece2b8b6272f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:18.530794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3269f80a-cd11-471b-9a4d-ece2b8b6272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:18.594599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 584/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:18.864238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5932-8097-4ccd-802f-bc65c041fbc3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:18.961906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5932-8097-4ccd-802f-bc65c041fbc3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:19.000930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a571cdd-fa46-4873-be3a-c55cd2782f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:19.001533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a571cdd-fa46-4873-be3a-c55cd2782f6e None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:19.001814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a571cdd-fa46-4873-be3a-c55cd2782f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:19.002117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a571cdd-fa46-4873-be3a-c55cd2782f6e None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:19.002788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a571cdd-fa46-4873-be3a-c55cd2782f6e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:19.004151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a571cdd-fa46-4873-be3a-c55cd2782f6e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:19.019511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 585/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Wed Sep 21 05:16:18 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:16:21.169174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875060d7-34e2-40a8-a419-1cf4e1e62415 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:21.169702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875060d7-34e2-40a8-a419-1cf4e1e62415 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:21.169702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875060d7-34e2-40a8-a419-1cf4e1e62415 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:21.171074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 585/1170] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:21.176448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c4723-ae43-4bc3-8677-ad9e8ab9cc84 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:21.176835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c4723-ae43-4bc3-8677-ad9e8ab9cc84 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:21.177005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c4723-ae43-4bc3-8677-ad9e8ab9cc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:21.194717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-845c4723-ae43-4bc3-8677-ad9e8ab9cc84 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:21.195932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-845c4723-ae43-4bc3-8677-ad9e8ab9cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:21.199149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:21.262044 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 586/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:21.534885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875060d7-34e2-40a8-a419-1cf4e1e62415 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:21.634994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875060d7-34e2-40a8-a419-1cf4e1e62415 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:21.650433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:21.667506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed65c4-dd05-4d4f-ade9-b7c5f19e581c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:21.668118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed65c4-dd05-4d4f-ade9-b7c5f19e581c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:21.668392 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed65c4-dd05-4d4f-ade9-b7c5f19e581c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:21.668660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed65c4-dd05-4d4f-ade9-b7c5f19e581c None admin] PATH_INFO: `/v3/domains/8a778705ed0b46f58b41d3dc74868c57` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:21.678521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed65c4-dd05-4d4f-ade9-b7c5f19e581c None admin] RBAC: Authorizing `identity:get_domain(domain_id=8a778705ed0b46f58b41d3dc74868c57)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:21.679930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed65c4-dd05-4d4f-ade9-b7c5f19e581c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:21.682034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 586/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Sep 21 05:16:21 2022] GET /identity/v3/domains/8a778705ed0b46f58b41d3dc74868c57 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:21.687983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c4723-ae43-4bc3-8677-ad9e8ab9cc84 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:21.781912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c4723-ae43-4bc3-8677-ad9e8ab9cc84 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:21.814214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71651cf9-37a2-4192-ae68-ff4bab8935e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:21.814748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71651cf9-37a2-4192-ae68-ff4bab8935e7 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:21.815026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71651cf9-37a2-4192-ae68-ff4bab8935e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:21.815303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71651cf9-37a2-4192-ae68-ff4bab8935e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:21.815947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71651cf9-37a2-4192-ae68-ff4bab8935e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:21.817356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71651cf9-37a2-4192-ae68-ff4bab8935e7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:21.828403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:21.844120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 587/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:16:21 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:16:23.993597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45def5-93db-43f4-8e7b-27384c2924bc None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:23.994130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45def5-93db-43f4-8e7b-27384c2924bc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:23.994130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45def5-93db-43f4-8e7b-27384c2924bc None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:23.995423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 587/1174] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:24.000728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd7453-c318-44ad-ae02-0295e0f296b7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:24.001126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd7453-c318-44ad-ae02-0295e0f296b7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:24.001286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd7453-c318-44ad-ae02-0295e0f296b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:24.018734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-40dd7453-c318-44ad-ae02-0295e0f296b7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:24.019905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-40dd7453-c318-44ad-ae02-0295e0f296b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:24.084236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 588/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:24.356099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45def5-93db-43f4-8e7b-27384c2924bc None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:24.454665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45def5-93db-43f4-8e7b-27384c2924bc None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:24.487696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15951d52-7de3-4d8c-a157-13f9f0d58fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:24.488314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15951d52-7de3-4d8c-a157-13f9f0d58fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:24.488595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15951d52-7de3-4d8c-a157-13f9f0d58fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:24.488871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15951d52-7de3-4d8c-a157-13f9f0d58fa0 None admin] PATH_INFO: `/v3/domains/8a778705ed0b46f58b41d3dc74868c57` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:24.489991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15951d52-7de3-4d8c-a157-13f9f0d58fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8a778705ed0b46f58b41d3dc74868c57)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:24.491413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15951d52-7de3-4d8c-a157-13f9f0d58fa0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:24.493472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 588/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Sep 21 05:16:24 2022] GET /identity/v3/domains/8a778705ed0b46f58b41d3dc74868c57 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:24.499059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd7453-c318-44ad-ae02-0295e0f296b7 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:24.594810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd7453-c318-44ad-ae02-0295e0f296b7 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:24.617661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:24.629956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa735a08-a823-4111-9b44-7c2651c64cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:24.630557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa735a08-a823-4111-9b44-7c2651c64cee None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:24.630835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa735a08-a823-4111-9b44-7c2651c64cee None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:24.631105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa735a08-a823-4111-9b44-7c2651c64cee None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:24.631743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa735a08-a823-4111-9b44-7c2651c64cee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:24.633153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa735a08-a823-4111-9b44-7c2651c64cee None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:24.654209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 589/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:16:24 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:16:26.816427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772047-23b4-4020-b999-4d9b75d78aa1 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:26.816961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772047-23b4-4020-b999-4d9b75d78aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:26.816961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772047-23b4-4020-b999-4d9b75d78aa1 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:26.818344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 589/1178] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:26.823513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a399f61-f446-44d3-8c40-40a98a581a0b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:26.823877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a399f61-f446-44d3-8c40-40a98a581a0b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:26.824059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a399f61-f446-44d3-8c40-40a98a581a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:26.842180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4a399f61-f446-44d3-8c40-40a98a581a0b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:26.843295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4a399f61-f446-44d3-8c40-40a98a581a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:26.906909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 590/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:27.178783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9772047-23b4-4020-b999-4d9b75d78aa1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:27.226441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:27.280507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9772047-23b4-4020-b999-4d9b75d78aa1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:27.313169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e95b10-68e4-4da9-8fca-0aba96d1ec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:27.313788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e95b10-68e4-4da9-8fca-0aba96d1ec7c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:27.314098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e95b10-68e4-4da9-8fca-0aba96d1ec7c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:27.314375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e95b10-68e4-4da9-8fca-0aba96d1ec7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:27.324617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e95b10-68e4-4da9-8fca-0aba96d1ec7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-79e95b10-68e4-4da9-8fca-0aba96d1ec7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:16:27.329326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:16:27.330431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:16:27.331266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 590/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:16:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:16:27.336456 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a399f61-f446-44d3-8c40-40a98a581a0b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:27.433769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a399f61-f446-44d3-8c40-40a98a581a0b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:27.442478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:27.468254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa77edf-c6e1-4d8f-becd-e9265fc7f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:27.468853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77edf-c6e1-4d8f-becd-e9265fc7f748 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:27.469127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77edf-c6e1-4d8f-becd-e9265fc7f748 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:27.469396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77edf-c6e1-4d8f-becd-e9265fc7f748 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:27.470170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77edf-c6e1-4d8f-becd-e9265fc7f748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:27.470485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77edf-c6e1-4d8f-becd-e9265fc7f748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:27.471833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77edf-c6e1-4d8f-becd-e9265fc7f748 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:27.479636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 591/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:16:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:27.485591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e95b10-68e4-4da9-8fca-0aba96d1ec7c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:27.582393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e95b10-68e4-4da9-8fca-0aba96d1ec7c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:27.584687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:27.616680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8975b-59b0-49dd-b3e7-3fb131fb30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:27.617303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8975b-59b0-49dd-b3e7-3fb131fb30d0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:27.617585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8975b-59b0-49dd-b3e7-3fb131fb30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:27.618393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8975b-59b0-49dd-b3e7-3fb131fb30d0 None admin] PATH_INFO: `/v3/users/348ead193980493e8498ad330afd92f4` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:27.631010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8975b-59b0-49dd-b3e7-3fb131fb30d0 None admin] RBAC: Authorizing `identity:get_user(user_id=348ead193980493e8498ad330afd92f4)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:27.632441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8975b-59b0-49dd-b3e7-3fb131fb30d0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:27.634684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 591/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:16:27 2022] GET /identity/v3/users/348ead193980493e8498ad330afd92f4 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:27.640603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa77edf-c6e1-4d8f-becd-e9265fc7f748 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:27.736242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa77edf-c6e1-4d8f-becd-e9265fc7f748 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:27.769195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626d2ab-b598-4eba-8a12-ac31712e2e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:27.769786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626d2ab-b598-4eba-8a12-ac31712e2e6a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:27.770096 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626d2ab-b598-4eba-8a12-ac31712e2e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:27.770369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626d2ab-b598-4eba-8a12-ac31712e2e6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:27.807493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:27.836898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626d2ab-b598-4eba-8a12-ac31712e2e6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:27.838370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626d2ab-b598-4eba-8a12-ac31712e2e6a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:27.933403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 592/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:16:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:16:27.939269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8975b-59b0-49dd-b3e7-3fb131fb30d0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:28.033830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8975b-59b0-49dd-b3e7-3fb131fb30d0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:28.066734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a58dc7-ae13-4a73-aa43-c8902147dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:28.067350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a58dc7-ae13-4a73-aa43-c8902147dc97 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:28.067629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a58dc7-ae13-4a73-aa43-c8902147dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:28.067900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a58dc7-ae13-4a73-aa43-c8902147dc97 None admin] PATH_INFO: `/v3/projects/a62e8d0baa084b07b36e10f46efe76e5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:28.068992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a58dc7-ae13-4a73-aa43-c8902147dc97 None admin] RBAC: Authorizing `identity:get_project(project_id=a62e8d0baa084b07b36e10f46efe76e5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:28.070431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a58dc7-ae13-4a73-aa43-c8902147dc97 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:28.072027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 592/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:16:27 2022] GET /identity/v3/projects/a62e8d0baa084b07b36e10f46efe76e5 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:28.077831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626d2ab-b598-4eba-8a12-ac31712e2e6a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:28.144589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:28.174984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626d2ab-b598-4eba-8a12-ac31712e2e6a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:28.207846 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac978fc6-39ac-4c34-804b-e00a2d6ab1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:28.208438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac978fc6-39ac-4c34-804b-e00a2d6ab1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:28.208718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac978fc6-39ac-4c34-804b-e00a2d6ab1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:28.208991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac978fc6-39ac-4c34-804b-e00a2d6ab1d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:28.209705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac978fc6-39ac-4c34-804b-e00a2d6ab1d4 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=a62e8d0baa084b07b36e10f46efe76e5, user.id=348ead193980493e8498ad330afd92f4) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:28.210039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac978fc6-39ac-4c34-804b-e00a2d6ab1d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:28.212060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac978fc6-39ac-4c34-804b-e00a2d6ab1d4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:28.217223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 593/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:16:28 2022] GET /identity/v3/role_assignments?user.id=348ead193980493e8498ad330afd92f4&scope.project.id=a62e8d0baa084b07b36e10f46efe76e5&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:30.303557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4caab3a-1af3-48c7-96fd-f65c0ff63f98 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:30.304081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4caab3a-1af3-48c7-96fd-f65c0ff63f98 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:30.304081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4caab3a-1af3-48c7-96fd-f65c0ff63f98 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:30.306055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 593/1186] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:30.311104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1857c3-123b-4a93-809a-8ba57a192f75 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:30.311569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1857c3-123b-4a93-809a-8ba57a192f75 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:30.311913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1857c3-123b-4a93-809a-8ba57a192f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:30.330671 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5f1857c3-123b-4a93-809a-8ba57a192f75 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:30.331867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f1857c3-123b-4a93-809a-8ba57a192f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:30.395258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 594/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:30.659507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4caab3a-1af3-48c7-96fd-f65c0ff63f98 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:30.688962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:30.757775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4caab3a-1af3-48c7-96fd-f65c0ff63f98 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:30.790223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337fe99-46c7-48c3-a413-a6d5556ac004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:30.790838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337fe99-46c7-48c3-a413-a6d5556ac004 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:30.791114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337fe99-46c7-48c3-a413-a6d5556ac004 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:30.791382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337fe99-46c7-48c3-a413-a6d5556ac004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:30.800918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337fe99-46c7-48c3-a413-a6d5556ac004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3337fe99-46c7-48c3-a413-a6d5556ac004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:16:30.805622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:16:30.806772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:16:30.808003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 594/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:16:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:16:30.812641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1857c3-123b-4a93-809a-8ba57a192f75 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:30.871574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:30.907888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1857c3-123b-4a93-809a-8ba57a192f75 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:30.939613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3477752-2b5e-46e5-970c-fdc329f8f6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:30.940204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3477752-2b5e-46e5-970c-fdc329f8f6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:30.940482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3477752-2b5e-46e5-970c-fdc329f8f6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:30.940750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3477752-2b5e-46e5-970c-fdc329f8f6e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:30.941402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3477752-2b5e-46e5-970c-fdc329f8f6e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:30.941702 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3477752-2b5e-46e5-970c-fdc329f8f6e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:30.943083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3477752-2b5e-46e5-970c-fdc329f8f6e3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:30.951475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 595/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:16:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:16:30.958892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337fe99-46c7-48c3-a413-a6d5556ac004 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:31.048648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:31.058278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337fe99-46c7-48c3-a413-a6d5556ac004 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:31.091460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c5230-f1d1-4345-88b3-0c0c3b8e59f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:31.092075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c5230-f1d1-4345-88b3-0c0c3b8e59f0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:31.092368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c5230-f1d1-4345-88b3-0c0c3b8e59f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:31.092628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c5230-f1d1-4345-88b3-0c0c3b8e59f0 None admin] PATH_INFO: `/v3/users/348ead193980493e8498ad330afd92f4` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:31.093722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c5230-f1d1-4345-88b3-0c0c3b8e59f0 None admin] RBAC: Authorizing `identity:get_user(user_id=348ead193980493e8498ad330afd92f4)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:31.095174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c5230-f1d1-4345-88b3-0c0c3b8e59f0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:31.096726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 595/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:16:30 2022] GET /identity/v3/users/348ead193980493e8498ad330afd92f4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:31.102472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3477752-2b5e-46e5-970c-fdc329f8f6e3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:31.198288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3477752-2b5e-46e5-970c-fdc329f8f6e3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:31.222875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:31.231412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1bf39-5769-4a3b-97c0-b373e07ed7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:31.232026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1bf39-5769-4a3b-97c0-b373e07ed7ac None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:31.232305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1bf39-5769-4a3b-97c0-b373e07ed7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:31.232577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1bf39-5769-4a3b-97c0-b373e07ed7ac None admin] PATH_INFO: `/v3/projects/a62e8d0baa084b07b36e10f46efe76e5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:31.233678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1bf39-5769-4a3b-97c0-b373e07ed7ac None admin] RBAC: Authorizing `identity:get_project(project_id=a62e8d0baa084b07b36e10f46efe76e5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:31.235129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1bf39-5769-4a3b-97c0-b373e07ed7ac None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:31.236723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 596/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:16:31 2022] GET /identity/v3/projects/a62e8d0baa084b07b36e10f46efe76e5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:31.243264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c5230-f1d1-4345-88b3-0c0c3b8e59f0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:31.340552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c5230-f1d1-4345-88b3-0c0c3b8e59f0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:31.373355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a969a5b-3f2e-482b-8654-95fea8662b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:31.374017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a969a5b-3f2e-482b-8654-95fea8662b29 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:31.374296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a969a5b-3f2e-482b-8654-95fea8662b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:31.374565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a969a5b-3f2e-482b-8654-95fea8662b29 None admin] PATH_INFO: `/v3/projects/a62e8d0baa084b07b36e10f46efe76e5/users/348ead193980493e8498ad330afd92f4/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:31.376407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a969a5b-3f2e-482b-8654-95fea8662b29 None admin] RBAC: Authorizing `identity:create_grant(project_id=a62e8d0baa084b07b36e10f46efe76e5, user_id=348ead193980493e8498ad330afd92f4, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:31.377813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a969a5b-3f2e-482b-8654-95fea8662b29 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:31.390386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 596/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:16:31 2022] PUT /identity/v3/projects/a62e8d0baa084b07b36e10f46efe76e5/users/348ead193980493e8498ad330afd92f4/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:16:33.496936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3db3f-51dc-42c3-a869-05236d1a15ce None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:33.497531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3db3f-51dc-42c3-a869-05236d1a15ce None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:33.497531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3db3f-51dc-42c3-a869-05236d1a15ce None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:33.498944 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 597/1193] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:16:33.504031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64599db7-b18a-4e32-9d3a-664cf5a2df8f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:33.504404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64599db7-b18a-4e32-9d3a-664cf5a2df8f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:33.504581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64599db7-b18a-4e32-9d3a-664cf5a2df8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:33.522164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-64599db7-b18a-4e32-9d3a-664cf5a2df8f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:16:33.523323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-64599db7-b18a-4e32-9d3a-664cf5a2df8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:16:33.536436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:33.589661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 597/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:16:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:16:33.861876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3db3f-51dc-42c3-a869-05236d1a15ce None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:33.959568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3db3f-51dc-42c3-a869-05236d1a15ce None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:33.991965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0841c-7756-4927-b7c0-d1c4442ecaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:33.992606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0841c-7756-4927-b7c0-d1c4442ecaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:33.992887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0841c-7756-4927-b7c0-d1c4442ecaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:33.993156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0841c-7756-4927-b7c0-d1c4442ecaa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:34.007987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0841c-7756-4927-b7c0-d1c4442ecaa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0de0841c-7756-4927-b7c0-d1c4442ecaa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:16:34.013259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:16:34.014366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:16:34.015417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 598/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:16:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:16:34.020564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64599db7-b18a-4e32-9d3a-664cf5a2df8f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:34.114486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64599db7-b18a-4e32-9d3a-664cf5a2df8f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:34.147050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318e371-6e3d-4f76-a298-57bda12e70ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:34.147640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e371-6e3d-4f76-a298-57bda12e70ff None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:34.147915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e371-6e3d-4f76-a298-57bda12e70ff None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:34.148182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e371-6e3d-4f76-a298-57bda12e70ff None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:34.149468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318e371-6e3d-4f76-a298-57bda12e70ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:34.149772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318e371-6e3d-4f76-a298-57bda12e70ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:34.151136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318e371-6e3d-4f76-a298-57bda12e70ff None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:34.159181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 598/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:16:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:34.165150 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0841c-7756-4927-b7c0-d1c4442ecaa5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:34.219068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:34.262895 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0841c-7756-4927-b7c0-d1c4442ecaa5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:34.299681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac035045-bfce-4eff-aa81-757e94a5423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:34.300304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac035045-bfce-4eff-aa81-757e94a5423a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:34.300583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac035045-bfce-4eff-aa81-757e94a5423a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:34.300855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac035045-bfce-4eff-aa81-757e94a5423a None admin] PATH_INFO: `/v3/users/348ead193980493e8498ad330afd92f4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:34.301986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac035045-bfce-4eff-aa81-757e94a5423a None admin] RBAC: Authorizing `identity:get_user(user_id=348ead193980493e8498ad330afd92f4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:34.303417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac035045-bfce-4eff-aa81-757e94a5423a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:34.304981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 599/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Sep 21 05:16:34 2022] GET /identity/v3/users/348ead193980493e8498ad330afd92f4 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:34.311290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318e371-6e3d-4f76-a298-57bda12e70ff None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:34.324291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:34.408548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318e371-6e3d-4f76-a298-57bda12e70ff None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:34.440216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b0c0f-f766-4155-ad45-d97aa5875c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:34.440808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b0c0f-f766-4155-ad45-d97aa5875c35 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:34.441084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b0c0f-f766-4155-ad45-d97aa5875c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:34.441354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b0c0f-f766-4155-ad45-d97aa5875c35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:34.505290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b0c0f-f766-4155-ad45-d97aa5875c35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:34.506720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b0c0f-f766-4155-ad45-d97aa5875c35 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:34.517993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:34.603490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 599/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:16:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:16:34.609352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac035045-bfce-4eff-aa81-757e94a5423a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:34.705021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac035045-bfce-4eff-aa81-757e94a5423a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:34.726421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:34.738510 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b225f7-b321-403b-b771-7a420f3b8216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:34.739118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b225f7-b321-403b-b771-7a420f3b8216 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:34.739393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b225f7-b321-403b-b771-7a420f3b8216 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:34.739663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b225f7-b321-403b-b771-7a420f3b8216 None admin] PATH_INFO: `/v3/projects/a62e8d0baa084b07b36e10f46efe76e5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:34.740744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b225f7-b321-403b-b771-7a420f3b8216 None admin] RBAC: Authorizing `identity:get_project(project_id=a62e8d0baa084b07b36e10f46efe76e5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:34.742175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b225f7-b321-403b-b771-7a420f3b8216 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:34.744359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 600/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Sep 21 05:16:34 2022] GET /identity/v3/projects/a62e8d0baa084b07b36e10f46efe76e5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:34.750473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b0c0f-f766-4155-ad45-d97aa5875c35 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:16:34.844232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b0c0f-f766-4155-ad45-d97aa5875c35 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:16:34.855554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:34.878809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30944d2c-c983-43e1-9440-cbf0d9745e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:16:34.879393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30944d2c-c983-43e1-9440-cbf0d9745e67 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:16:34.879667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30944d2c-c983-43e1-9440-cbf0d9745e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:16:34.879934 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30944d2c-c983-43e1-9440-cbf0d9745e67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:16:34.880631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30944d2c-c983-43e1-9440-cbf0d9745e67 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=a62e8d0baa084b07b36e10f46efe76e5, user.id=348ead193980493e8498ad330afd92f4) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:16:34.880958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30944d2c-c983-43e1-9440-cbf0d9745e67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:16:34.882387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30944d2c-c983-43e1-9440-cbf0d9745e67 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:16:34.887807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 600/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:16:34 2022] GET /identity/v3/role_assignments?user.id=348ead193980493e8498ad330afd92f4&scope.project.id=a62e8d0baa084b07b36e10f46efe76e5&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:16:44.734250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:16:44.885429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=13 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:30.288760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc49c-e6d3-40fc-955d-b7c7fae684ac None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:30.289360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc49c-e6d3-40fc-955d-b7c7fae684ac None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:30.289360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc49c-e6d3-40fc-955d-b7c7fae684ac None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:30.290884 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 601/1201] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:17:30.296973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdef81-a2f5-4b45-b64b-d992873dc583 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:30.297576 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdef81-a2f5-4b45-b64b-d992873dc583 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:30.297576 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdef81-a2f5-4b45-b64b-d992873dc583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:30.318998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4efdef81-a2f5-4b45-b64b-d992873dc583 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:17:30.321291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4efdef81-a2f5-4b45-b64b-d992873dc583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:17:30.410217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 601/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:17:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:17:30.714468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfc49c-e6d3-40fc-955d-b7c7fae684ac None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:30.847726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfc49c-e6d3-40fc-955d-b7c7fae684ac None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:30.889914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d25d7-08ee-4b0f-b6c4-423292317c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:30.890719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d25d7-08ee-4b0f-b6c4-423292317c76 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:30.891081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d25d7-08ee-4b0f-b6c4-423292317c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:30.891341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d25d7-08ee-4b0f-b6c4-423292317c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:30.903939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d25d7-08ee-4b0f-b6c4-423292317c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:30.905586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d25d7-08ee-4b0f-b6c4-423292317c76 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-537d25d7-08ee-4b0f-b6c4-423292317c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:30.923762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:30.924893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:30.925864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:30.925864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:30.925864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:30.930103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 602/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:30.932976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdef81-a2f5-4b45-b64b-d992873dc583 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:31.064183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdef81-a2f5-4b45-b64b-d992873dc583 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:31.096569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:31.105872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5da1e2-9c35-437d-9970-acbf803f0981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:31.106568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5da1e2-9c35-437d-9970-acbf803f0981 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:31.106848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5da1e2-9c35-437d-9970-acbf803f0981 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:31.107123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5da1e2-9c35-437d-9970-acbf803f0981 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:31.107854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5da1e2-9c35-437d-9970-acbf803f0981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:31.108315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5da1e2-9c35-437d-9970-acbf803f0981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:31.111310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5da1e2-9c35-437d-9970-acbf803f0981 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:31.133425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 602/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:31.133643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d25d7-08ee-4b0f-b6c4-423292317c76 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:31.200340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:31.267131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d25d7-08ee-4b0f-b6c4-423292317c76 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:31.312665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a5614-f081-45a6-897e-b8b2c64b7024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:31.313331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a5614-f081-45a6-897e-b8b2c64b7024 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:31.313609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a5614-f081-45a6-897e-b8b2c64b7024 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:31.313881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a5614-f081-45a6-897e-b8b2c64b7024 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:31.314647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a5614-f081-45a6-897e-b8b2c64b7024 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:31.316226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a5614-f081-45a6-897e-b8b2c64b7024 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:31.340746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 603/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Sep 21 05:17:31 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:17:33.736094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d0894-031e-4cec-807c-89b395e2a4b5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:33.736660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d0894-031e-4cec-807c-89b395e2a4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:33.736660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d0894-031e-4cec-807c-89b395e2a4b5 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:33.738409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 603/1206] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:17:33.744502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab76ec-96c9-472e-ba60-b6b21875b4c1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:33.744983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab76ec-96c9-472e-ba60-b6b21875b4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:33.745070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab76ec-96c9-472e-ba60-b6b21875b4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:33.764600 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-beab76ec-96c9-472e-ba60-b6b21875b4c1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:17:33.765888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-beab76ec-96c9-472e-ba60-b6b21875b4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:17:33.845978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 604/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:17:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 05:17:34.118839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d0894-031e-4cec-807c-89b395e2a4b5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:34.236723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d0894-031e-4cec-807c-89b395e2a4b5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:34.278315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8ea9c-c479-47fc-877f-79e485691646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:34.278974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8ea9c-c479-47fc-877f-79e485691646 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:34.279255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8ea9c-c479-47fc-877f-79e485691646 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:34.279534 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8ea9c-c479-47fc-877f-79e485691646 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:34.291486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8ea9c-c479-47fc-877f-79e485691646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-83e8ea9c-c479-47fc-877f-79e485691646 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:17:34.296866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:34.298011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:34.298871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 604/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:17:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:34.304256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab76ec-96c9-472e-ba60-b6b21875b4c1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:34.360210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:34.429196 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab76ec-96c9-472e-ba60-b6b21875b4c1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:34.468808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04543879-3db4-4dde-b644-92678602de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:34.469739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04543879-3db4-4dde-b644-92678602de4e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:34.470153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04543879-3db4-4dde-b644-92678602de4e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:34.470360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04543879-3db4-4dde-b644-92678602de4e None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:34.471202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04543879-3db4-4dde-b644-92678602de4e None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:34.471518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04543879-3db4-4dde-b644-92678602de4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:34.473215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04543879-3db4-4dde-b644-92678602de4e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:34.482320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 605/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:17:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:17:34.489067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8ea9c-c479-47fc-877f-79e485691646 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:34.549817 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:34.609234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8ea9c-c479-47fc-877f-79e485691646 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:34.651916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19ee3-bd9c-49a1-b23a-fa1eaefa2ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:34.652720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19ee3-bd9c-49a1-b23a-fa1eaefa2ebd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:34.653090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19ee3-bd9c-49a1-b23a-fa1eaefa2ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:34.653500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19ee3-bd9c-49a1-b23a-fa1eaefa2ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:34.665316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d19ee3-bd9c-49a1-b23a-fa1eaefa2ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:34.667073 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d19ee3-bd9c-49a1-b23a-fa1eaefa2ebd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a1d19ee3-bd9c-49a1-b23a-fa1eaefa2ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:34.684708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:34.685811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:34.686953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:34.686953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:34.686953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:34.686953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 605/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:34.691521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04543879-3db4-4dde-b644-92678602de4e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:34.809512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:34.812670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04543879-3db4-4dde-b644-92678602de4e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:34.854580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c77326-07ec-452d-8134-7ff93a042f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:34.855448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c77326-07ec-452d-8134-7ff93a042f37 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:34.855698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c77326-07ec-452d-8134-7ff93a042f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:34.856016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c77326-07ec-452d-8134-7ff93a042f37 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:34.856719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c77326-07ec-452d-8134-7ff93a042f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:34.857058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c77326-07ec-452d-8134-7ff93a042f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:34.858652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c77326-07ec-452d-8134-7ff93a042f37 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:34.877827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 606/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:34.879208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19ee3-bd9c-49a1-b23a-fa1eaefa2ebd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:34.998236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19ee3-bd9c-49a1-b23a-fa1eaefa2ebd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:34.999830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:35.041993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5456e-9bd1-4838-a13a-f6f13022627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:35.041993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5456e-9bd1-4838-a13a-f6f13022627a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:35.041993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5456e-9bd1-4838-a13a-f6f13022627a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:35.041993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5456e-9bd1-4838-a13a-f6f13022627a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:35.050243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5456e-9bd1-4838-a13a-f6f13022627a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:35.051850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5456e-9bd1-4838-a13a-f6f13022627a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-35b5456e-9bd1-4838-a13a-f6f13022627a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:17:35.064024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:35.065192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:35.066193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:35.068029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 606/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:35.074653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c77326-07ec-452d-8134-7ff93a042f37 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:35.186186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c77326-07ec-452d-8134-7ff93a042f37 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:35.224486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9595-671e-4e2d-822c-6f1cb57609bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:35.225318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9595-671e-4e2d-822c-6f1cb57609bf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:35.225636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9595-671e-4e2d-822c-6f1cb57609bf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:35.225990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9595-671e-4e2d-822c-6f1cb57609bf None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:35.226869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d9595-671e-4e2d-822c-6f1cb57609bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:35.227229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d9595-671e-4e2d-822c-6f1cb57609bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:35.228683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d9595-671e-4e2d-822c-6f1cb57609bf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:35.249766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 607/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Sep 21 05:17:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:35.255942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5456e-9bd1-4838-a13a-f6f13022627a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:35.383794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5456e-9bd1-4838-a13a-f6f13022627a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:35.422872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecc4ad-cfa9-47d4-b0e4-39bf75bd0bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:35.423479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecc4ad-cfa9-47d4-b0e4-39bf75bd0bfe None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:35.423762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecc4ad-cfa9-47d4-b0e4-39bf75bd0bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:35.424039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecc4ad-cfa9-47d4-b0e4-39bf75bd0bfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:35.456047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:35.509859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecc4ad-cfa9-47d4-b0e4-39bf75bd0bfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:35.509859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecc4ad-cfa9-47d4-b0e4-39bf75bd0bfe None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:35.626901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 607/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:17:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:17:35.633243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9595-671e-4e2d-822c-6f1cb57609bf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:35.665846 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:35.782678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9595-671e-4e2d-822c-6f1cb57609bf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:35.818803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caa7c8-1dea-48be-a601-23ce34025b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:35.819616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa7c8-1dea-48be-a601-23ce34025b74 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:35.820542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa7c8-1dea-48be-a601-23ce34025b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:35.820676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa7c8-1dea-48be-a601-23ce34025b74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:35.832000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48caa7c8-1dea-48be-a601-23ce34025b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:35.833504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48caa7c8-1dea-48be-a601-23ce34025b74 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-48caa7c8-1dea-48be-a601-23ce34025b74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:35.848900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:35.854140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:35.855125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:35.855125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:35.855125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:35.855125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 608/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:35.856794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecc4ad-cfa9-47d4-b0e4-39bf75bd0bfe None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:35.931373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:35.979608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecc4ad-cfa9-47d4-b0e4-39bf75bd0bfe None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:36.020870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f4f31-7ea6-4fb7-bb53-b063cc88ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:36.021602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f4f31-7ea6-4fb7-bb53-b063cc88ea04 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:36.021892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f4f31-7ea6-4fb7-bb53-b063cc88ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:36.022259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f4f31-7ea6-4fb7-bb53-b063cc88ea04 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:36.022981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f4f31-7ea6-4fb7-bb53-b063cc88ea04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:36.023310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f4f31-7ea6-4fb7-bb53-b063cc88ea04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:36.024819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f4f31-7ea6-4fb7-bb53-b063cc88ea04 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:36.038680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 608/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:36.044181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caa7c8-1dea-48be-a601-23ce34025b74 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:36.127821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:36.165433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caa7c8-1dea-48be-a601-23ce34025b74 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:36.208926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c799e7-33a0-4986-9fbf-758136edb78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:36.209711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c799e7-33a0-4986-9fbf-758136edb78e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:36.210255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c799e7-33a0-4986-9fbf-758136edb78e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:36.210644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c799e7-33a0-4986-9fbf-758136edb78e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:36.222545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c799e7-33a0-4986-9fbf-758136edb78e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:36.224011 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c799e7-33a0-4986-9fbf-758136edb78e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-30c799e7-33a0-4986-9fbf-758136edb78e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:17:36.237830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:36.239024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:36.240152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:17:36.240152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:17:36.240152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:17:36.240152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:17:36.240152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:17:36.240152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:17:36.240152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:36.240152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:36.240152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 609/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:17:36.248444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f4f31-7ea6-4fb7-bb53-b063cc88ea04 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:36.364731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f4f31-7ea6-4fb7-bb53-b063cc88ea04 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:36.389226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:36.403240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbb6a8-8fba-4584-a7f7-b4d94b8c764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:36.403874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbb6a8-8fba-4584-a7f7-b4d94b8c764c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:36.404149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbb6a8-8fba-4584-a7f7-b4d94b8c764c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:36.404416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbb6a8-8fba-4584-a7f7-b4d94b8c764c None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:36.405106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbb6a8-8fba-4584-a7f7-b4d94b8c764c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:36.405403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbb6a8-8fba-4584-a7f7-b4d94b8c764c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:36.406833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbb6a8-8fba-4584-a7f7-b4d94b8c764c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:36.420157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 609/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:36.426923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c799e7-33a0-4986-9fbf-758136edb78e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:36.530504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c799e7-33a0-4986-9fbf-758136edb78e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:36.560296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:36.566598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7040-da9f-4dcf-85b7-766e9d81a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:36.567340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7040-da9f-4dcf-85b7-766e9d81a558 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:36.567660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7040-da9f-4dcf-85b7-766e9d81a558 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:36.568145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7040-da9f-4dcf-85b7-766e9d81a558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:36.568928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7040-da9f-4dcf-85b7-766e9d81a558 None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=4b3d8a1c8b6640ae9e90ece0bbfbf410) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:36.569345 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7040-da9f-4dcf-85b7-766e9d81a558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:36.570825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7040-da9f-4dcf-85b7-766e9d81a558 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:36.579203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 610/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:17:36 2022] GET /identity/v3/role_assignments?user.id=4b3d8a1c8b6640ae9e90ece0bbfbf410&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:38.795134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c7ccc-cbf1-4498-a9ae-f1c649ea0d8b None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:38.795783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c7ccc-cbf1-4498-a9ae-f1c649ea0d8b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:38.795783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c7ccc-cbf1-4498-a9ae-f1c649ea0d8b None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:38.797169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 610/1220] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:17:38.802862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27572-c925-4364-858d-ac8c823cef34 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:38.803416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27572-c925-4364-858d-ac8c823cef34 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:38.803547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27572-c925-4364-858d-ac8c823cef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:38.826446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-10b27572-c925-4364-858d-ac8c823cef34 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:17:38.827758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-10b27572-c925-4364-858d-ac8c823cef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:17:38.900519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 611/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:17:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:17:39.186894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c7ccc-cbf1-4498-a9ae-f1c649ea0d8b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:39.306632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c7ccc-cbf1-4498-a9ae-f1c649ea0d8b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:39.345404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fcecd-f7a3-4aa0-9734-9c4f0d9611bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:39.346066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fcecd-f7a3-4aa0-9734-9c4f0d9611bb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:39.346378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fcecd-f7a3-4aa0-9734-9c4f0d9611bb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:39.346625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fcecd-f7a3-4aa0-9734-9c4f0d9611bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:39.358596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fcecd-f7a3-4aa0-9734-9c4f0d9611bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a50fcecd-f7a3-4aa0-9734-9c4f0d9611bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:17:39.364559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:39.365715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:39.367357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 611/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:17:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:39.372963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27572-c925-4364-858d-ac8c823cef34 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:39.485033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27572-c925-4364-858d-ac8c823cef34 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:39.512501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:39.524020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f9b9a-d29b-49ad-9c67-9ed91e25d812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:39.524630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f9b9a-d29b-49ad-9c67-9ed91e25d812 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:39.524911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f9b9a-d29b-49ad-9c67-9ed91e25d812 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:39.525185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f9b9a-d29b-49ad-9c67-9ed91e25d812 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:39.525842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f9b9a-d29b-49ad-9c67-9ed91e25d812 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:39.526180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f9b9a-d29b-49ad-9c67-9ed91e25d812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:39.527580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f9b9a-d29b-49ad-9c67-9ed91e25d812 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:39.536135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 612/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:17:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:17:39.542353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fcecd-f7a3-4aa0-9734-9c4f0d9611bb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:39.587551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:39.651622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fcecd-f7a3-4aa0-9734-9c4f0d9611bb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:39.687100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de4231-5d0c-4710-b5e5-33198dfda087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:39.687732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de4231-5d0c-4710-b5e5-33198dfda087 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:39.688011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de4231-5d0c-4710-b5e5-33198dfda087 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:39.688282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de4231-5d0c-4710-b5e5-33198dfda087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:39.699331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de4231-5d0c-4710-b5e5-33198dfda087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:39.700802 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de4231-5d0c-4710-b5e5-33198dfda087 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-34de4231-5d0c-4710-b5e5-33198dfda087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:39.717103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:39.718346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:39.719482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:39.719482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:39.719482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:39.719482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 612/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:39.724414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f9b9a-d29b-49ad-9c67-9ed91e25d812 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:39.838741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f9b9a-d29b-49ad-9c67-9ed91e25d812 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:39.876875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9a935-e328-4999-8bcb-e1176a33cd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:39.877563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9a935-e328-4999-8bcb-e1176a33cd24 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:39.877854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9a935-e328-4999-8bcb-e1176a33cd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:39.878418 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9a935-e328-4999-8bcb-e1176a33cd24 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:39.879137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9a935-e328-4999-8bcb-e1176a33cd24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:39.879573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9a935-e328-4999-8bcb-e1176a33cd24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:39.881039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9a935-e328-4999-8bcb-e1176a33cd24 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:39.894980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 613/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:39.902441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de4231-5d0c-4710-b5e5-33198dfda087 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:40.004406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:40.012336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de4231-5d0c-4710-b5e5-33198dfda087 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:40.049311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0570bc0-f1cd-416e-890c-539ab7635de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:40.050532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0570bc0-f1cd-416e-890c-539ab7635de2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:40.050882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0570bc0-f1cd-416e-890c-539ab7635de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:40.051164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0570bc0-f1cd-416e-890c-539ab7635de2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:40.059488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0570bc0-f1cd-416e-890c-539ab7635de2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:40.061104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0570bc0-f1cd-416e-890c-539ab7635de2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-e0570bc0-f1cd-416e-890c-539ab7635de2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:17:40.073639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:40.079681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:40.081244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 613/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:40.082075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9a935-e328-4999-8bcb-e1176a33cd24 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:40.135030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:40.194039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9a935-e328-4999-8bcb-e1176a33cd24 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:40.234181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbbf59-0745-49cc-a353-6a07da9dbae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:40.234788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbbf59-0745-49cc-a353-6a07da9dbae9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:40.235774 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbbf59-0745-49cc-a353-6a07da9dbae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:40.235774 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbbf59-0745-49cc-a353-6a07da9dbae9 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:40.236412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbbf59-0745-49cc-a353-6a07da9dbae9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:40.236560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbbf59-0745-49cc-a353-6a07da9dbae9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:40.238549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbbf59-0745-49cc-a353-6a07da9dbae9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:40.258955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 614/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Sep 21 05:17:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:40.264971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0570bc0-f1cd-416e-890c-539ab7635de2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:40.376734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0570bc0-f1cd-416e-890c-539ab7635de2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:40.425026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1b5c4-bbf6-4457-93e6-b1f22ada74c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:40.425576 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1b5c4-bbf6-4457-93e6-b1f22ada74c1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:40.425887 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1b5c4-bbf6-4457-93e6-b1f22ada74c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:40.426201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1b5c4-bbf6-4457-93e6-b1f22ada74c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:40.440602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1b5c4-bbf6-4457-93e6-b1f22ada74c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:40.441978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1b5c4-bbf6-4457-93e6-b1f22ada74c1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-9fb1b5c4-bbf6-4457-93e6-b1f22ada74c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:40.457789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:40.459539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:40.460636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:40.460636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:40.460636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:40.460636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 614/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:17:40.465183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbbf59-0745-49cc-a353-6a07da9dbae9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:40.572718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbbf59-0745-49cc-a353-6a07da9dbae9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:40.575386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:40.615324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551eaa-1e13-4325-9eec-bce9f5eeffa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:40.615936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551eaa-1e13-4325-9eec-bce9f5eeffa0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:40.616214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551eaa-1e13-4325-9eec-bce9f5eeffa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:40.617101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551eaa-1e13-4325-9eec-bce9f5eeffa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:40.617720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85551eaa-1e13-4325-9eec-bce9f5eeffa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:40.618073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85551eaa-1e13-4325-9eec-bce9f5eeffa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:40.619447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85551eaa-1e13-4325-9eec-bce9f5eeffa0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:40.632332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 615/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:40.638535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1b5c4-bbf6-4457-93e6-b1f22ada74c1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:40.648249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:40.754007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1b5c4-bbf6-4457-93e6-b1f22ada74c1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:40.791337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7976c5b-ae84-4427-bcc7-bd479b39027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:40.791971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7976c5b-ae84-4427-bcc7-bd479b39027b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:40.792255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7976c5b-ae84-4427-bcc7-bd479b39027b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:40.792530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7976c5b-ae84-4427-bcc7-bd479b39027b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:40.800036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7976c5b-ae84-4427-bcc7-bd479b39027b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:40.801558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7976c5b-ae84-4427-bcc7-bd479b39027b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b7976c5b-ae84-4427-bcc7-bd479b39027b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:17:40.814043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:40.815373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:40.816610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:17:40.816610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:17:40.816610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:17:40.816610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:17:40.816610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:17:40.816610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:17:40.816610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:40.816610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:40.817720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 615/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:40.824394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551eaa-1e13-4325-9eec-bce9f5eeffa0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:40.933738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551eaa-1e13-4325-9eec-bce9f5eeffa0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:40.971298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19e3d8-cae0-44ca-ad02-136daf3aa5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:40.971968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19e3d8-cae0-44ca-ad02-136daf3aa5ec None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:40.972327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19e3d8-cae0-44ca-ad02-136daf3aa5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:40.972637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19e3d8-cae0-44ca-ad02-136daf3aa5ec None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:40.973387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19e3d8-cae0-44ca-ad02-136daf3aa5ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:40.973698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19e3d8-cae0-44ca-ad02-136daf3aa5ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:40.975334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19e3d8-cae0-44ca-ad02-136daf3aa5ec None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:40.995273 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 616/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:40.996621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7976c5b-ae84-4427-bcc7-bd479b39027b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:41.084479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:41.121874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7976c5b-ae84-4427-bcc7-bd479b39027b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:41.158561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade771f-ae87-4ff9-868b-78981aabfaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:41.159221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade771f-ae87-4ff9-868b-78981aabfaa1 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:41.159503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade771f-ae87-4ff9-868b-78981aabfaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:41.159775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade771f-ae87-4ff9-868b-78981aabfaa1 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/4b3d8a1c8b6640ae9e90ece0bbfbf410/roles/99269efe3b2a430e9cc7912268d2be84` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:41.174120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade771f-ae87-4ff9-868b-78981aabfaa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=4b3d8a1c8b6640ae9e90ece0bbfbf410, role_id=99269efe3b2a430e9cc7912268d2be84)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:41.175559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade771f-ae87-4ff9-868b-78981aabfaa1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:41.189015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 616/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:17:40 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/4b3d8a1c8b6640ae9e90ece0bbfbf410/roles/99269efe3b2a430e9cc7912268d2be84 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:17:43.541223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3918d-0876-4277-81b4-75307cb5098d None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:43.541822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3918d-0876-4277-81b4-75307cb5098d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:43.541822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3918d-0876-4277-81b4-75307cb5098d None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:43.543238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 617/1233] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:17:43.549200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0025e0-99fb-44b8-8135-a21036fc4040 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:43.549629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0025e0-99fb-44b8-8135-a21036fc4040 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:43.549818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0025e0-99fb-44b8-8135-a21036fc4040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:43.570225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3b0025e0-99fb-44b8-8135-a21036fc4040 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:17:43.571418 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3b0025e0-99fb-44b8-8135-a21036fc4040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:17:43.641202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 617/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:17:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:17:43.917343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3918d-0876-4277-81b4-75307cb5098d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:43.943670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:44.032912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3918d-0876-4277-81b4-75307cb5098d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:44.072072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02561a-30f0-4db3-913b-6ed2aed43db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:44.072700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02561a-30f0-4db3-913b-6ed2aed43db1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:44.072976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02561a-30f0-4db3-913b-6ed2aed43db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:44.074018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02561a-30f0-4db3-913b-6ed2aed43db1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:44.085824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02561a-30f0-4db3-913b-6ed2aed43db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5d02561a-30f0-4db3-913b-6ed2aed43db1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:17:44.091191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:44.092367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 05:17:44.093255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:44.097518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 618/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Sep 21 05:17:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:44.098651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0025e0-99fb-44b8-8135-a21036fc4040 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:44.149154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:44.207512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0025e0-99fb-44b8-8135-a21036fc4040 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:44.245426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc91886-56a7-43ef-8467-70b2f86d6357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:44.246074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc91886-56a7-43ef-8467-70b2f86d6357 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:44.246377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc91886-56a7-43ef-8467-70b2f86d6357 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:44.246661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc91886-56a7-43ef-8467-70b2f86d6357 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:44.247366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc91886-56a7-43ef-8467-70b2f86d6357 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:44.247731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc91886-56a7-43ef-8467-70b2f86d6357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:44.249143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc91886-56a7-43ef-8467-70b2f86d6357 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:44.259136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 618/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Sep 21 05:17:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:44.265777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02561a-30f0-4db3-913b-6ed2aed43db1 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:44.368433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:44.384595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02561a-30f0-4db3-913b-6ed2aed43db1 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:44.419518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091ae57-8b0d-4168-9fc2-1c743058bb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:44.420146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091ae57-8b0d-4168-9fc2-1c743058bb02 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:44.420422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091ae57-8b0d-4168-9fc2-1c743058bb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:44.420697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091ae57-8b0d-4168-9fc2-1c743058bb02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:44.431616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091ae57-8b0d-4168-9fc2-1c743058bb02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:44.433693 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091ae57-8b0d-4168-9fc2-1c743058bb02 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-1091ae57-8b0d-4168-9fc2-1c743058bb02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:44.448789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:44.450051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:44.450969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:44.450969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:44.450969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:44.450969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 619/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:44.455934 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc91886-56a7-43ef-8467-70b2f86d6357 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:44.563506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc91886-56a7-43ef-8467-70b2f86d6357 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:44.573666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:44.602655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770cfe7-9181-44ca-8289-aab803ef6dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:44.603367 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770cfe7-9181-44ca-8289-aab803ef6dfa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:44.603680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770cfe7-9181-44ca-8289-aab803ef6dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:44.604016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770cfe7-9181-44ca-8289-aab803ef6dfa None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:44.604681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770cfe7-9181-44ca-8289-aab803ef6dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:44.604984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770cfe7-9181-44ca-8289-aab803ef6dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:44.606510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770cfe7-9181-44ca-8289-aab803ef6dfa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:44.620403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 619/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:44.626469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091ae57-8b0d-4168-9fc2-1c743058bb02 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:44.733871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091ae57-8b0d-4168-9fc2-1c743058bb02 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:44.770427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cbb84-055b-4544-8ef3-7a5e3ffb9201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:44.771113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cbb84-055b-4544-8ef3-7a5e3ffb9201 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:44.771493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cbb84-055b-4544-8ef3-7a5e3ffb9201 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:44.771849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cbb84-055b-4544-8ef3-7a5e3ffb9201 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:44.775362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:44.781067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cbb84-055b-4544-8ef3-7a5e3ffb9201 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:44.783080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cbb84-055b-4544-8ef3-7a5e3ffb9201 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-993cbb84-055b-4544-8ef3-7a5e3ffb9201 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:17:44.796225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:44.797649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:44.798873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:44.802111 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 620/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:44.803723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770cfe7-9181-44ca-8289-aab803ef6dfa None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:44.916782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770cfe7-9181-44ca-8289-aab803ef6dfa None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:44.954368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a5efd-a1ce-48ee-8284-c2823e084a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:44.955100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5efd-a1ce-48ee-8284-c2823e084a55 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:44.955382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5efd-a1ce-48ee-8284-c2823e084a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:44.955655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5efd-a1ce-48ee-8284-c2823e084a55 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:44.956375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a5efd-a1ce-48ee-8284-c2823e084a55 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:44.956678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a5efd-a1ce-48ee-8284-c2823e084a55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:44.958191 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a5efd-a1ce-48ee-8284-c2823e084a55 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:44.977188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 620/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Sep 21 05:17:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:17:44.983716 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cbb84-055b-4544-8ef3-7a5e3ffb9201 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:45.091869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cbb84-055b-4544-8ef3-7a5e3ffb9201 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:45.126834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfd4a8-4be9-403f-ad6c-d33502086d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:45.127972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfd4a8-4be9-403f-ad6c-d33502086d91 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:45.127972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfd4a8-4be9-403f-ad6c-d33502086d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:45.127972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfd4a8-4be9-403f-ad6c-d33502086d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:45.192984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:45.200942 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfd4a8-4be9-403f-ad6c-d33502086d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:45.202638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfd4a8-4be9-403f-ad6c-d33502086d91 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:45.314428 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 621/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:17:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:17:45.321319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a5efd-a1ce-48ee-8284-c2823e084a55 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:45.352851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:45.437514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a5efd-a1ce-48ee-8284-c2823e084a55 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:45.473744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c0106-0949-40a3-9226-4763f4da7c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:45.474333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c0106-0949-40a3-9226-4763f4da7c96 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:45.474641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c0106-0949-40a3-9226-4763f4da7c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:45.474944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c0106-0949-40a3-9226-4763f4da7c96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:45.486169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c0106-0949-40a3-9226-4763f4da7c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:45.488437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c0106-0949-40a3-9226-4763f4da7c96 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-048c0106-0949-40a3-9226-4763f4da7c96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:45.506210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:45.507979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:45.508899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:45.508899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:45.508899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:45.508899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 621/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:17:45.514189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfd4a8-4be9-403f-ad6c-d33502086d91 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:45.617892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:45.624634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfd4a8-4be9-403f-ad6c-d33502086d91 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:45.662203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cebdc-9af6-4bff-be6d-561d9adc5961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:45.662966 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cebdc-9af6-4bff-be6d-561d9adc5961 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:45.663438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cebdc-9af6-4bff-be6d-561d9adc5961 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:45.663826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cebdc-9af6-4bff-be6d-561d9adc5961 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:45.664632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09cebdc-9af6-4bff-be6d-561d9adc5961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:45.665006 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09cebdc-9af6-4bff-be6d-561d9adc5961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:45.666563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09cebdc-9af6-4bff-be6d-561d9adc5961 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:45.678546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 622/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:45.684756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c0106-0949-40a3-9226-4763f4da7c96 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:45.777903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:45.804685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c0106-0949-40a3-9226-4763f4da7c96 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:45.844146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27832-bad1-4596-9f62-fe8d579fccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:45.844798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27832-bad1-4596-9f62-fe8d579fccf9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:45.845071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27832-bad1-4596-9f62-fe8d579fccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:45.845352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27832-bad1-4596-9f62-fe8d579fccf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:45.852910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d27832-bad1-4596-9f62-fe8d579fccf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:45.854420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d27832-bad1-4596-9f62-fe8d579fccf9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c3d27832-bad1-4596-9f62-fe8d579fccf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:17:45.867695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:45.868805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:45.869681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:17:45.869681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:17:45.869681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:17:45.869681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:17:45.869681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:17:45.869681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:17:45.869681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:45.869681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:45.869681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 622/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:17:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:17:45.874943 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cebdc-9af6-4bff-be6d-561d9adc5961 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:45.986714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cebdc-9af6-4bff-be6d-561d9adc5961 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:46.023976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fba77-d7de-40a9-8e92-9240a0aae251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:46.024587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fba77-d7de-40a9-8e92-9240a0aae251 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:46.024870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fba77-d7de-40a9-8e92-9240a0aae251 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:46.025144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fba77-d7de-40a9-8e92-9240a0aae251 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:46.025830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fba77-d7de-40a9-8e92-9240a0aae251 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:46.026169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fba77-d7de-40a9-8e92-9240a0aae251 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:46.027668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fba77-d7de-40a9-8e92-9240a0aae251 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:46.040376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 623/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:17:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:46.046618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27832-bad1-4596-9f62-fe8d579fccf9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:46.164781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27832-bad1-4596-9f62-fe8d579fccf9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:46.208118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19db420-6ae1-41ed-9561-5006d971ab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:46.208762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19db420-6ae1-41ed-9561-5006d971ab83 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:46.209042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19db420-6ae1-41ed-9561-5006d971ab83 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:46.209312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19db420-6ae1-41ed-9561-5006d971ab83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:46.210145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19db420-6ae1-41ed-9561-5006d971ab83 None admin] RBAC: Adding query filter params (role.id=99269efe3b2a430e9cc7912268d2be84, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=4b3d8a1c8b6640ae9e90ece0bbfbf410) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:46.210454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19db420-6ae1-41ed-9561-5006d971ab83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:46.211928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19db420-6ae1-41ed-9561-5006d971ab83 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:46.218483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 623/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:17:46 2022] GET /identity/v3/role_assignments?user.id=4b3d8a1c8b6640ae9e90ece0bbfbf410&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=99269efe3b2a430e9cc7912268d2be84 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:48.526479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638534d-3666-4107-bad3-fe0abdaec49f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:48.527054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638534d-3666-4107-bad3-fe0abdaec49f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:48.527054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638534d-3666-4107-bad3-fe0abdaec49f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:48.528356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 624/1247] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:17:48.534080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca102c-c39c-4142-b645-411ba73babd3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:48.534598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca102c-c39c-4142-b645-411ba73babd3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:48.534598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca102c-c39c-4142-b645-411ba73babd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:48.541832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:48.556662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-eaca102c-c39c-4142-b645-411ba73babd3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:17:48.557889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-eaca102c-c39c-4142-b645-411ba73babd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:17:48.628140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 624/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:17:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:17:48.900385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638534d-3666-4107-bad3-fe0abdaec49f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:48.919143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:49.010568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638534d-3666-4107-bad3-fe0abdaec49f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:49.048570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb1300-4cda-4355-85bf-50b271376813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:49.049390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb1300-4cda-4355-85bf-50b271376813 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:49.049715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb1300-4cda-4355-85bf-50b271376813 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:49.050016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb1300-4cda-4355-85bf-50b271376813 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:49.061012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb1300-4cda-4355-85bf-50b271376813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-cfdb1300-4cda-4355-85bf-50b271376813 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:17:49.067288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:49.068611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:49.070217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 625/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:17:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:49.074306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca102c-c39c-4142-b645-411ba73babd3 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:49.195009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca102c-c39c-4142-b645-411ba73babd3 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:49.229660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:49.236229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9ae45-0edd-4df9-952f-9e5e28d8cdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:49.236960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9ae45-0edd-4df9-952f-9e5e28d8cdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:49.237671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9ae45-0edd-4df9-952f-9e5e28d8cdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:49.238135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9ae45-0edd-4df9-952f-9e5e28d8cdf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:49.238837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9ae45-0edd-4df9-952f-9e5e28d8cdf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:49.239150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9ae45-0edd-4df9-952f-9e5e28d8cdf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:49.240666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9ae45-0edd-4df9-952f-9e5e28d8cdf2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:49.252319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 625/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:49.258621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb1300-4cda-4355-85bf-50b271376813 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:49.346128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:49.373793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb1300-4cda-4355-85bf-50b271376813 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:49.408312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fcb10-dcc6-4b60-b233-5a53027e53fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:49.408936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fcb10-dcc6-4b60-b233-5a53027e53fa None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:49.409212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fcb10-dcc6-4b60-b233-5a53027e53fa None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:49.409480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fcb10-dcc6-4b60-b233-5a53027e53fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:49.420168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fcb10-dcc6-4b60-b233-5a53027e53fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:49.421753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fcb10-dcc6-4b60-b233-5a53027e53fa None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-1b1fcb10-dcc6-4b60-b233-5a53027e53fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:49.437119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:49.438557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:49.439999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:49.439999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:49.439999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:49.439999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 626/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:49.445501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9ae45-0edd-4df9-952f-9e5e28d8cdf2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:49.556493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9ae45-0edd-4df9-952f-9e5e28d8cdf2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:49.597903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d4daf-b706-4acb-a89c-9712674ebdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:49.598555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d4daf-b706-4acb-a89c-9712674ebdaa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:49.598836 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d4daf-b706-4acb-a89c-9712674ebdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:49.599107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d4daf-b706-4acb-a89c-9712674ebdaa None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:49.599756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d4daf-b706-4acb-a89c-9712674ebdaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:49.600062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d4daf-b706-4acb-a89c-9712674ebdaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:49.601968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d4daf-b706-4acb-a89c-9712674ebdaa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:49.615185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 626/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:49.621583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fcb10-dcc6-4b60-b233-5a53027e53fa None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:49.726210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fcb10-dcc6-4b60-b233-5a53027e53fa None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:49.754180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:49.762598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190dd42-5d39-445e-815d-95fbb8ba3f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:49.763227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190dd42-5d39-445e-815d-95fbb8ba3f2f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:49.763500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190dd42-5d39-445e-815d-95fbb8ba3f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:49.763764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190dd42-5d39-445e-815d-95fbb8ba3f2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:49.771615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190dd42-5d39-445e-815d-95fbb8ba3f2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:49.773106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190dd42-5d39-445e-815d-95fbb8ba3f2f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-b190dd42-5d39-445e-815d-95fbb8ba3f2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:17:49.784822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:49.786215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:49.787260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 627/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:49.792495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d4daf-b706-4acb-a89c-9712674ebdaa None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:49.851138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:49.898399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d4daf-b706-4acb-a89c-9712674ebdaa None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:49.933667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33349b-3e25-4ffd-9515-4e783c7689ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:49.934311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33349b-3e25-4ffd-9515-4e783c7689ca None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:49.934595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33349b-3e25-4ffd-9515-4e783c7689ca None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:49.934868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33349b-3e25-4ffd-9515-4e783c7689ca None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:49.935594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33349b-3e25-4ffd-9515-4e783c7689ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:49.935900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33349b-3e25-4ffd-9515-4e783c7689ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:49.937330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33349b-3e25-4ffd-9515-4e783c7689ca None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:49.955457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 627/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Sep 21 05:17:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:49.961336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190dd42-5d39-445e-815d-95fbb8ba3f2f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:50.061498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190dd42-5d39-445e-815d-95fbb8ba3f2f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:50.095357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031087e-feb7-49e6-93ae-a28497d1a4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:50.095969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031087e-feb7-49e6-93ae-a28497d1a4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:50.096254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031087e-feb7-49e6-93ae-a28497d1a4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:50.096530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031087e-feb7-49e6-93ae-a28497d1a4f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:50.154951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:50.168285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031087e-feb7-49e6-93ae-a28497d1a4f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:50.169724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031087e-feb7-49e6-93ae-a28497d1a4f9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:50.273246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 628/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:17:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:17:50.279095 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33349b-3e25-4ffd-9515-4e783c7689ca None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:50.374383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33349b-3e25-4ffd-9515-4e783c7689ca None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:50.387693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:50.407962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342566-652a-4914-8511-b3d0bf664908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:50.408576 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342566-652a-4914-8511-b3d0bf664908 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:50.408853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342566-652a-4914-8511-b3d0bf664908 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:50.409124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342566-652a-4914-8511-b3d0bf664908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:50.419231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0342566-652a-4914-8511-b3d0bf664908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:50.420649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0342566-652a-4914-8511-b3d0bf664908 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f0342566-652a-4914-8511-b3d0bf664908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:50.435133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:50.436210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:50.437090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:50.437090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:50.437090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:50.437090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 628/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:50.441804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031087e-feb7-49e6-93ae-a28497d1a4f9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:50.525212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:50.537071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031087e-feb7-49e6-93ae-a28497d1a4f9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:50.570928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b1a8e-f5e6-48bc-9415-470c8d33c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:50.571519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b1a8e-f5e6-48bc-9415-470c8d33c816 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:50.571795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b1a8e-f5e6-48bc-9415-470c8d33c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:50.572066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b1a8e-f5e6-48bc-9415-470c8d33c816 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:50.572690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b1a8e-f5e6-48bc-9415-470c8d33c816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:50.572989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b1a8e-f5e6-48bc-9415-470c8d33c816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:50.574371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b1a8e-f5e6-48bc-9415-470c8d33c816 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:50.585068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 629/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:50.591428 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342566-652a-4914-8511-b3d0bf664908 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:50.688103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342566-652a-4914-8511-b3d0bf664908 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:50.720639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f16f9-0415-4ad8-b8b4-ba5cfdf5ed28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:50.721257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f16f9-0415-4ad8-b8b4-ba5cfdf5ed28 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:50.721537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f16f9-0415-4ad8-b8b4-ba5cfdf5ed28 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:50.721812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f16f9-0415-4ad8-b8b4-ba5cfdf5ed28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:50.728864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f16f9-0415-4ad8-b8b4-ba5cfdf5ed28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:50.730309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f16f9-0415-4ad8-b8b4-ba5cfdf5ed28 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c00f16f9-0415-4ad8-b8b4-ba5cfdf5ed28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:17:50.741461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:50.742504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:50.743606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:17:50.743606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:17:50.743606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:17:50.743606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:17:50.743606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:17:50.743606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:17:50.743606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:50.743606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:50.743606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 629/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:17:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:50.748382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b1a8e-f5e6-48bc-9415-470c8d33c816 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:50.844279 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b1a8e-f5e6-48bc-9415-470c8d33c816 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:50.876898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc30e4-ccda-4a84-b078-f4f89ec26732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:50.877483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc30e4-ccda-4a84-b078-f4f89ec26732 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:50.877760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc30e4-ccda-4a84-b078-f4f89ec26732 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:50.878060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc30e4-ccda-4a84-b078-f4f89ec26732 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:50.878735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc30e4-ccda-4a84-b078-f4f89ec26732 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:50.879036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc30e4-ccda-4a84-b078-f4f89ec26732 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:50.880442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc30e4-ccda-4a84-b078-f4f89ec26732 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:50.891930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 630/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:17:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:50.898483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f16f9-0415-4ad8-b8b4-ba5cfdf5ed28 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:50.923294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:50.999226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f16f9-0415-4ad8-b8b4-ba5cfdf5ed28 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:51.031327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766edfe-5872-4286-aa22-c4183ad152fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:51.031917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766edfe-5872-4286-aa22-c4183ad152fb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:51.032191 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766edfe-5872-4286-aa22-c4183ad152fb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:51.032458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766edfe-5872-4286-aa22-c4183ad152fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:51.033158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766edfe-5872-4286-aa22-c4183ad152fb None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=4b3d8a1c8b6640ae9e90ece0bbfbf410) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:51.033457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766edfe-5872-4286-aa22-c4183ad152fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:51.034885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766edfe-5872-4286-aa22-c4183ad152fb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:51.040545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 630/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:17:50 2022] GET /identity/v3/role_assignments?user.id=4b3d8a1c8b6640ae9e90ece0bbfbf410&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:53.128446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3deb1c-580f-4b22-a03a-721646c7fb28 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:53.128976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3deb1c-580f-4b22-a03a-721646c7fb28 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:53.128976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3deb1c-580f-4b22-a03a-721646c7fb28 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:53.130340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 631/1261] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:17:53.135554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384d5a4-ef26-45ec-b3cc-86f19f3f10fc None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:53.135959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384d5a4-ef26-45ec-b3cc-86f19f3f10fc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:53.136108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384d5a4-ef26-45ec-b3cc-86f19f3f10fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:53.154472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3384d5a4-ef26-45ec-b3cc-86f19f3f10fc None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:17:53.155657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3384d5a4-ef26-45ec-b3cc-86f19f3f10fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:17:53.221875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 631/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:17:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:17:53.517182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3deb1c-580f-4b22-a03a-721646c7fb28 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:53.528491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:53.616224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3deb1c-580f-4b22-a03a-721646c7fb28 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:53.649747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce0ebf-74cd-4659-9ccb-212f3ab22f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:53.650400 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce0ebf-74cd-4659-9ccb-212f3ab22f9f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:53.650675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce0ebf-74cd-4659-9ccb-212f3ab22f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:53.650944 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce0ebf-74cd-4659-9ccb-212f3ab22f9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:53.660681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce0ebf-74cd-4659-9ccb-212f3ab22f9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-adce0ebf-74cd-4659-9ccb-212f3ab22f9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:17:53.665395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:53.666505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:53.667704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 632/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:17:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:53.672495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384d5a4-ef26-45ec-b3cc-86f19f3f10fc None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:53.690662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:53.768359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384d5a4-ef26-45ec-b3cc-86f19f3f10fc None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:53.800278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3a7ec-de22-4859-8edd-190325be5fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:53.800869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3a7ec-de22-4859-8edd-190325be5fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:53.801149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3a7ec-de22-4859-8edd-190325be5fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:53.801424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3a7ec-de22-4859-8edd-190325be5fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:53.802102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3a7ec-de22-4859-8edd-190325be5fa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:53.802410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3a7ec-de22-4859-8edd-190325be5fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:53.804293 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3a7ec-de22-4859-8edd-190325be5fa4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:53.812000 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 632/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:17:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:17:53.817472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce0ebf-74cd-4659-9ccb-212f3ab22f9f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:53.881472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:53.913872 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce0ebf-74cd-4659-9ccb-212f3ab22f9f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:53.946796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc061615-0da0-467b-93c1-a1ecb784584b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:53.947396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc061615-0da0-467b-93c1-a1ecb784584b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:53.947674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc061615-0da0-467b-93c1-a1ecb784584b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:53.947944 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc061615-0da0-467b-93c1-a1ecb784584b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:53.957454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc061615-0da0-467b-93c1-a1ecb784584b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:53.958899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc061615-0da0-467b-93c1-a1ecb784584b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-cc061615-0da0-467b-93c1-a1ecb784584b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:53.972722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:53.973809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:53.974949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:53.974949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:53.974949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:53.974949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 633/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:53.979569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3a7ec-de22-4859-8edd-190325be5fa4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:54.058129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:54.077183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3a7ec-de22-4859-8edd-190325be5fa4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:54.109411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda024f9-c9f6-4485-9811-dd455449eb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:54.110019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda024f9-c9f6-4485-9811-dd455449eb93 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:54.110300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda024f9-c9f6-4485-9811-dd455449eb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:54.110570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda024f9-c9f6-4485-9811-dd455449eb93 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:54.111199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda024f9-c9f6-4485-9811-dd455449eb93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:54.111498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda024f9-c9f6-4485-9811-dd455449eb93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:54.113086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda024f9-c9f6-4485-9811-dd455449eb93 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:54.124891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 633/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:54.130502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc061615-0da0-467b-93c1-a1ecb784584b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:54.228739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc061615-0da0-467b-93c1-a1ecb784584b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:54.252103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:54.266313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41c430-bc52-4980-a2ad-7cafbec8aacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:54.266929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41c430-bc52-4980-a2ad-7cafbec8aacc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:54.267207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41c430-bc52-4980-a2ad-7cafbec8aacc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:54.267480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41c430-bc52-4980-a2ad-7cafbec8aacc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:54.275460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41c430-bc52-4980-a2ad-7cafbec8aacc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:54.276889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41c430-bc52-4980-a2ad-7cafbec8aacc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3a41c430-bc52-4980-a2ad-7cafbec8aacc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:17:54.288231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:54.289324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:54.290313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 634/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:54.295199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda024f9-c9f6-4485-9811-dd455449eb93 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:54.390007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda024f9-c9f6-4485-9811-dd455449eb93 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:54.422468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1818cf9-3853-48ce-8ea1-1388ac02e58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:54.423057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1818cf9-3853-48ce-8ea1-1388ac02e58e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:54.423333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1818cf9-3853-48ce-8ea1-1388ac02e58e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:54.423601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1818cf9-3853-48ce-8ea1-1388ac02e58e None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:54.424308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1818cf9-3853-48ce-8ea1-1388ac02e58e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:54.424607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1818cf9-3853-48ce-8ea1-1388ac02e58e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:54.426018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1818cf9-3853-48ce-8ea1-1388ac02e58e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:54.434854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:54.443275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 634/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Sep 21 05:17:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:54.449008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41c430-bc52-4980-a2ad-7cafbec8aacc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:54.545754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41c430-bc52-4980-a2ad-7cafbec8aacc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:54.577331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0309923-1a80-4260-9570-1ddc43273570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:54.577960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0309923-1a80-4260-9570-1ddc43273570 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:54.578244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0309923-1a80-4260-9570-1ddc43273570 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:54.578516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0309923-1a80-4260-9570-1ddc43273570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:54.589377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0309923-1a80-4260-9570-1ddc43273570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:54.590856 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0309923-1a80-4260-9570-1ddc43273570 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-d0309923-1a80-4260-9570-1ddc43273570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:54.605362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:54.606492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:54.607732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:54.607732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:54.607732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:54.607732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 635/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:54.612106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1818cf9-3853-48ce-8ea1-1388ac02e58e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:54.707862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1818cf9-3853-48ce-8ea1-1388ac02e58e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:54.740879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db988e7f-de1b-44b8-9130-6a931da52cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:54.741474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db988e7f-de1b-44b8-9130-6a931da52cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:54.741751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db988e7f-de1b-44b8-9130-6a931da52cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:54.742052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db988e7f-de1b-44b8-9130-6a931da52cc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:54.742678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db988e7f-de1b-44b8-9130-6a931da52cc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:54.742981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db988e7f-de1b-44b8-9130-6a931da52cc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:54.744323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db988e7f-de1b-44b8-9130-6a931da52cc6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:54.754983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 635/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:54.760600 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0309923-1a80-4260-9570-1ddc43273570 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:54.791709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:54.857648 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0309923-1a80-4260-9570-1ddc43273570 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:54.890361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a0cc-c3d4-4c4d-aa94-6898b8529b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:54.890981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a0cc-c3d4-4c4d-aa94-6898b8529b80 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:54.891261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a0cc-c3d4-4c4d-aa94-6898b8529b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:54.891536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a0cc-c3d4-4c4d-aa94-6898b8529b80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:54.898500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66a0cc-c3d4-4c4d-aa94-6898b8529b80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:54.899925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66a0cc-c3d4-4c4d-aa94-6898b8529b80 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-bf66a0cc-c3d4-4c4d-aa94-6898b8529b80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:17:54.910820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:54.911844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:54.912993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:17:54.912993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:17:54.912993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:17:54.912993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:17:54.912993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:17:54.912993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:17:54.912993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:54.912993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:54.912993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 636/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:54.917738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db988e7f-de1b-44b8-9130-6a931da52cc6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:54.974310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:55.015216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db988e7f-de1b-44b8-9130-6a931da52cc6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:55.048245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88a4b3-65af-4245-a7c9-38fc55de119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:55.048834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88a4b3-65af-4245-a7c9-38fc55de119b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:55.049112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88a4b3-65af-4245-a7c9-38fc55de119b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:55.049383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88a4b3-65af-4245-a7c9-38fc55de119b None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:55.050100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88a4b3-65af-4245-a7c9-38fc55de119b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:55.050406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88a4b3-65af-4245-a7c9-38fc55de119b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:55.051800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88a4b3-65af-4245-a7c9-38fc55de119b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:55.063828 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 636/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:55.069645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a0cc-c3d4-4c4d-aa94-6898b8529b80 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:55.157857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:55.167322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a0cc-c3d4-4c4d-aa94-6898b8529b80 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:55.199786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3b85-25ca-48eb-bfbb-897c3f589455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:55.200427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3b85-25ca-48eb-bfbb-897c3f589455 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:55.200709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3b85-25ca-48eb-bfbb-897c3f589455 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:55.200984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3b85-25ca-48eb-bfbb-897c3f589455 None admin] PATH_INFO: `/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/4b3d8a1c8b6640ae9e90ece0bbfbf410/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:55.202863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e3b85-25ca-48eb-bfbb-897c3f589455 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c64d6cb0eb348a6a4a0718a7425d3ed, user_id=4b3d8a1c8b6640ae9e90ece0bbfbf410, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:55.204284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e3b85-25ca-48eb-bfbb-897c3f589455 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:55.216729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 637/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Sep 21 05:17:55 2022] PUT /identity/v3/projects/1c64d6cb0eb348a6a4a0718a7425d3ed/users/4b3d8a1c8b6640ae9e90ece0bbfbf410/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:17:57.365487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727f08-1f29-4585-a5e2-3696ec477fb2 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:57.366043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727f08-1f29-4585-a5e2-3696ec477fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:57.366233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727f08-1f29-4585-a5e2-3696ec477fb2 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:57.368439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 637/1274] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:17:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:17:57.374238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae9ac7-02b4-4d9b-856f-1fb0061e0e33 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:57.374688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae9ac7-02b4-4d9b-856f-1fb0061e0e33 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:57.374787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae9ac7-02b4-4d9b-856f-1fb0061e0e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:57.393404 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9dae9ac7-02b4-4d9b-856f-1fb0061e0e33 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:17:57.394614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9dae9ac7-02b4-4d9b-856f-1fb0061e0e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:17:57.461815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 638/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:17:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:17:57.726323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727f08-1f29-4585-a5e2-3696ec477fb2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:57.822841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727f08-1f29-4585-a5e2-3696ec477fb2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:57.839057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:57.855935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6e131-6850-42f7-9ae9-11cf38a777d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:57.856542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6e131-6850-42f7-9ae9-11cf38a777d5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:57.856819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6e131-6850-42f7-9ae9-11cf38a777d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:57.857086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6e131-6850-42f7-9ae9-11cf38a777d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:57.867550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6e131-6850-42f7-9ae9-11cf38a777d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-eda6e131-6850-42f7-9ae9-11cf38a777d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 05:17:57.872214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:57.873317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:57.874260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 638/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Sep 21 05:17:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 05:17:57.879112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae9ac7-02b4-4d9b-856f-1fb0061e0e33 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:57.955600 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:57.974127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae9ac7-02b4-4d9b-856f-1fb0061e0e33 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:58.008626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039525c-fe37-4f29-b4d7-8d4c1a87dd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:58.009165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039525c-fe37-4f29-b4d7-8d4c1a87dd0b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:58.009446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039525c-fe37-4f29-b4d7-8d4c1a87dd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:58.009720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039525c-fe37-4f29-b4d7-8d4c1a87dd0b None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:58.010401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039525c-fe37-4f29-b4d7-8d4c1a87dd0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:58.010706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039525c-fe37-4f29-b4d7-8d4c1a87dd0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:58.012077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039525c-fe37-4f29-b4d7-8d4c1a87dd0b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:58.019841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 639/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Sep 21 05:17:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:58.025528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6e131-6850-42f7-9ae9-11cf38a777d5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:58.123524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6e131-6850-42f7-9ae9-11cf38a777d5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:58.155292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7e7b2-15be-4ff1-9329-edd48a685291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:58.155901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e7b2-15be-4ff1-9329-edd48a685291 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:58.156182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e7b2-15be-4ff1-9329-edd48a685291 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:58.156457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e7b2-15be-4ff1-9329-edd48a685291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:58.167112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7e7b2-15be-4ff1-9329-edd48a685291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:58.168539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7e7b2-15be-4ff1-9329-edd48a685291 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1ba7e7b2-15be-4ff1-9329-edd48a685291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:58.182616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:58.183748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:58.184636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:58.184636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:58.184636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:58.184636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 639/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:58.189166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039525c-fe37-4f29-b4d7-8d4c1a87dd0b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:58.303130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039525c-fe37-4f29-b4d7-8d4c1a87dd0b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:58.335322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38656b7d-5cf4-45b7-9da1-aed7d9ccc54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:58.335913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38656b7d-5cf4-45b7-9da1-aed7d9ccc54b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:58.336189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38656b7d-5cf4-45b7-9da1-aed7d9ccc54b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:58.336459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38656b7d-5cf4-45b7-9da1-aed7d9ccc54b None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:58.337086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38656b7d-5cf4-45b7-9da1-aed7d9ccc54b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:58.337384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38656b7d-5cf4-45b7-9da1-aed7d9ccc54b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:58.338764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38656b7d-5cf4-45b7-9da1-aed7d9ccc54b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:58.349972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 640/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:58.356236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7e7b2-15be-4ff1-9329-edd48a685291 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:58.387050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:58.453250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7e7b2-15be-4ff1-9329-edd48a685291 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:58.485328 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe21623-89ec-4fab-92c9-8bc27f557402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:58.485956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe21623-89ec-4fab-92c9-8bc27f557402 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:58.486233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe21623-89ec-4fab-92c9-8bc27f557402 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:58.486497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe21623-89ec-4fab-92c9-8bc27f557402 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:58.494356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe21623-89ec-4fab-92c9-8bc27f557402 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:58.495757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe21623-89ec-4fab-92c9-8bc27f557402 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4fe21623-89ec-4fab-92c9-8bc27f557402 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 05:17:58.506998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:58.508131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:17:58.509292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 640/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:58.513766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38656b7d-5cf4-45b7-9da1-aed7d9ccc54b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:58.517518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:58.608391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38656b7d-5cf4-45b7-9da1-aed7d9ccc54b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:58.640800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617be5-4321-439f-a2ba-15868b24ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:58.641391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617be5-4321-439f-a2ba-15868b24ee23 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:58.641666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617be5-4321-439f-a2ba-15868b24ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:58.641945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617be5-4321-439f-a2ba-15868b24ee23 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:58.642667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f617be5-4321-439f-a2ba-15868b24ee23 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:58.642967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f617be5-4321-439f-a2ba-15868b24ee23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:58.644377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f617be5-4321-439f-a2ba-15868b24ee23 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:58.660434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 641/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Sep 21 05:17:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:58.666611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe21623-89ec-4fab-92c9-8bc27f557402 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:58.753220 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:58.763064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe21623-89ec-4fab-92c9-8bc27f557402 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:58.794961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27499c3c-119e-4316-8f5d-8ff792416fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:58.795548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27499c3c-119e-4316-8f5d-8ff792416fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:58.795825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27499c3c-119e-4316-8f5d-8ff792416fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:58.796093 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27499c3c-119e-4316-8f5d-8ff792416fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:58.859138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27499c3c-119e-4316-8f5d-8ff792416fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:58.860488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27499c3c-119e-4316-8f5d-8ff792416fd8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:58.943869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:58.953716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 641/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Sep 21 05:17:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 05:17:58.959412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617be5-4321-439f-a2ba-15868b24ee23 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:59.017986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:59.055036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617be5-4321-439f-a2ba-15868b24ee23 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:59.087229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad854c-3633-44db-bcd7-9e93317ab5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:59.087851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad854c-3633-44db-bcd7-9e93317ab5df None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:59.088126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad854c-3633-44db-bcd7-9e93317ab5df None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:59.088393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad854c-3633-44db-bcd7-9e93317ab5df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:59.098108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad854c-3633-44db-bcd7-9e93317ab5df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:59.099545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad854c-3633-44db-bcd7-9e93317ab5df None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c1ad854c-3633-44db-bcd7-9e93317ab5df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 05:17:59.114390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:59.115468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 05:17:59.116370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 05:17:59.116370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 05:17:59.116370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:59.116370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 642/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:59.120905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27499c3c-119e-4316-8f5d-8ff792416fd8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:59.214130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27499c3c-119e-4316-8f5d-8ff792416fd8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:59.248482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d00d0-18d0-496e-a4f5-1bebdc7f6f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:59.249069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d00d0-18d0-496e-a4f5-1bebdc7f6f68 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:59.249345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d00d0-18d0-496e-a4f5-1bebdc7f6f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:59.249611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d00d0-18d0-496e-a4f5-1bebdc7f6f68 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:59.250256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d00d0-18d0-496e-a4f5-1bebdc7f6f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:59.250553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d00d0-18d0-496e-a4f5-1bebdc7f6f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:59.251885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d00d0-18d0-496e-a4f5-1bebdc7f6f68 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:59.262478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 642/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Sep 21 05:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:59.267976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad854c-3633-44db-bcd7-9e93317ab5df None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:59.370362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad854c-3633-44db-bcd7-9e93317ab5df None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:59.384068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:59.405749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8552-41a8-428d-8da0-898a78cdea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:59.406393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8552-41a8-428d-8da0-898a78cdea96 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:59.406670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8552-41a8-428d-8da0-898a78cdea96 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:59.406940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8552-41a8-428d-8da0-898a78cdea96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:59.413335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b8552-41a8-428d-8da0-898a78cdea96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:59.414793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b8552-41a8-428d-8da0-898a78cdea96 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c64b8552-41a8-428d-8da0-898a78cdea96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 05:17:59.426461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:17:59.427581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:17:59.428841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:17:59.428841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:17:59.428841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:17:59.428841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:17:59.428841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:17:59.428841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:17:59.428841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 05:17:59.428841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Sep 21 05:17:59.428841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 643/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Sep 21 05:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:17:59.433383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d00d0-18d0-496e-a4f5-1bebdc7f6f68 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:59.483820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:17:59.532236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d00d0-18d0-496e-a4f5-1bebdc7f6f68 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:59.563700 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfba92-941c-4190-8d05-d6bf6e064009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:59.564290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfba92-941c-4190-8d05-d6bf6e064009 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:59.564565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfba92-941c-4190-8d05-d6bf6e064009 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:59.564831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfba92-941c-4190-8d05-d6bf6e064009 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:59.565497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfba92-941c-4190-8d05-d6bf6e064009 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:59.565796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfba92-941c-4190-8d05-d6bf6e064009 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:59.567219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfba92-941c-4190-8d05-d6bf6e064009 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:59.578985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 643/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Sep 21 05:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:17:59.585540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8552-41a8-428d-8da0-898a78cdea96 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:17:59.683440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8552-41a8-428d-8da0-898a78cdea96 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:17:59.715174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc33657-1438-4419-902b-60b6ab61c628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:17:59.715765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc33657-1438-4419-902b-60b6ab61c628 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:17:59.716043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc33657-1438-4419-902b-60b6ab61c628 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:17:59.716309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc33657-1438-4419-902b-60b6ab61c628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:17:59.717016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc33657-1438-4419-902b-60b6ab61c628 None admin] RBAC: Adding query filter params (role.id=51e166ed71fd44f8ab08c961b23a2118, scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed, user.id=4b3d8a1c8b6640ae9e90ece0bbfbf410) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:17:59.717320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc33657-1438-4419-902b-60b6ab61c628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:17:59.718749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc33657-1438-4419-902b-60b6ab61c628 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:17:59.724209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 644/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Sep 21 05:17:59 2022] GET /identity/v3/role_assignments?user.id=4b3d8a1c8b6640ae9e90ece0bbfbf410&scope.project.id=1c64d6cb0eb348a6a4a0718a7425d3ed&role.id=51e166ed71fd44f8ab08c961b23a2118 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:18:01.854838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c5bd-dd49-4b25-9124-3e519baff686 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:01.855356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c5bd-dd49-4b25-9124-3e519baff686 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:01.855356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c5bd-dd49-4b25-9124-3e519baff686 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:01.856618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 644/1288] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:18:01.861920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0e8bc-cc68-4184-a25b-ac2502403fd6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:01.862591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0e8bc-cc68-4184-a25b-ac2502403fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:01.862772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0e8bc-cc68-4184-a25b-ac2502403fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:01.880326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1fa0e8bc-cc68-4184-a25b-ac2502403fd6 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:18:01.882106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1fa0e8bc-cc68-4184-a25b-ac2502403fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:18:01.910552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:01.947098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 645/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:18:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:18:02.210431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c5bd-dd49-4b25-9124-3e519baff686 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:02.315377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c5bd-dd49-4b25-9124-3e519baff686 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:02.326165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:02.348382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8551aa-f3e3-492b-bc20-595853737f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:02.349005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8551aa-f3e3-492b-bc20-595853737f4e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:02.349283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8551aa-f3e3-492b-bc20-595853737f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:02.349560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8551aa-f3e3-492b-bc20-595853737f4e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:02.350688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8551aa-f3e3-492b-bc20-595853737f4e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:02.352025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8551aa-f3e3-492b-bc20-595853737f4e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4c8551aa-f3e3-492b-bc20-595853737f4e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:18:02.360651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:18:02.361913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:18:02.363131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 645/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:18:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:18:02.368698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0e8bc-cc68-4184-a25b-ac2502403fd6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:02.461903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0e8bc-cc68-4184-a25b-ac2502403fd6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:02.495056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053b85a-4a33-4e81-9d4e-13fddb8fb690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:02.495639 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053b85a-4a33-4e81-9d4e-13fddb8fb690 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:02.495913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053b85a-4a33-4e81-9d4e-13fddb8fb690 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:02.496178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053b85a-4a33-4e81-9d4e-13fddb8fb690 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:02.496793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053b85a-4a33-4e81-9d4e-13fddb8fb690 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:18:02.497089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053b85a-4a33-4e81-9d4e-13fddb8fb690 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:02.498468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053b85a-4a33-4e81-9d4e-13fddb8fb690 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:02.503083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 646/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Sep 21 05:18:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:18:02.508721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8551aa-f3e3-492b-bc20-595853737f4e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:02.604282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8551aa-f3e3-492b-bc20-595853737f4e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:02.636644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a5aad-cafa-4997-8e12-746b14cd2eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:02.637275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a5aad-cafa-4997-8e12-746b14cd2eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:02.637573 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a5aad-cafa-4997-8e12-746b14cd2eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:02.637846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a5aad-cafa-4997-8e12-746b14cd2eb1 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:02.638500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a5aad-cafa-4997-8e12-746b14cd2eb1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:18:02.638802 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a5aad-cafa-4997-8e12-746b14cd2eb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:02.640158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a5aad-cafa-4997-8e12-746b14cd2eb1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:02.644437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 646/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Sep 21 05:18:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:18:04.752824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16a77b-7f38-4a86-b68a-166265f5e5db None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:04.753348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16a77b-7f38-4a86-b68a-166265f5e5db None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:04.753348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16a77b-7f38-4a86-b68a-166265f5e5db None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:04.754735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 647/1293] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:18:04.760531 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975248a-faa6-4b89-9c4d-4077d3e97a89 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:04.760899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975248a-faa6-4b89-9c4d-4077d3e97a89 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:04.761082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975248a-faa6-4b89-9c4d-4077d3e97a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:04.779294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d975248a-faa6-4b89-9c4d-4077d3e97a89 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:18:04.780461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d975248a-faa6-4b89-9c4d-4077d3e97a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:18:04.808490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:04.845115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 647/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:18:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:18:05.112030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16a77b-7f38-4a86-b68a-166265f5e5db None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:05.143371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:05.212027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16a77b-7f38-4a86-b68a-166265f5e5db None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:05.247990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a66d5-d7a0-412a-9a6e-7add0acf95e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:05.248653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a66d5-d7a0-412a-9a6e-7add0acf95e6 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:05.248899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a66d5-d7a0-412a-9a6e-7add0acf95e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:05.249173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a66d5-d7a0-412a-9a6e-7add0acf95e6 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:05.249831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a66d5-d7a0-412a-9a6e-7add0acf95e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:05.251265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a66d5-d7a0-412a-9a6e-7add0acf95e6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:05.266214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 648/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Sep 21 05:18:05 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:18:07.393545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4251e4-8f29-4e96-9345-1b02be81fb0d None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:07.394107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4251e4-8f29-4e96-9345-1b02be81fb0d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:07.394107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4251e4-8f29-4e96-9345-1b02be81fb0d None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:07.395445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 648/1296] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:18:07.401226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7a18-9077-4ac5-b089-714f501bbff8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:07.401616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7a18-9077-4ac5-b089-714f501bbff8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:07.401782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7a18-9077-4ac5-b089-714f501bbff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:07.420125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1c1c7a18-9077-4ac5-b089-714f501bbff8 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:18:07.421313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1c1c7a18-9077-4ac5-b089-714f501bbff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:18:07.492963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 649/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:18:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:18:07.760287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4251e4-8f29-4e96-9345-1b02be81fb0d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:07.860812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4251e4-8f29-4e96-9345-1b02be81fb0d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:07.893611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a3434-024f-49eb-b7a6-c517ebd98dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:07.894256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a3434-024f-49eb-b7a6-c517ebd98dab None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:07.894536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a3434-024f-49eb-b7a6-c517ebd98dab None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:07.894805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a3434-024f-49eb-b7a6-c517ebd98dab None admin] PATH_INFO: `/v3/services/placement` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:07.895388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a3434-024f-49eb-b7a6-c517ebd98dab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:07.896752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a3434-024f-49eb-b7a6-c517ebd98dab None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5d6a3434-024f-49eb-b7a6-c517ebd98dab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:18:07.905375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:18:07.906571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:18:07.907530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 649/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:18:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:18:07.913045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7a18-9077-4ac5-b089-714f501bbff8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:07.931304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:08.009870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7a18-9077-4ac5-b089-714f501bbff8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:08.043878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cf781-51f1-4b78-8feb-2ae4f9f8ea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:08.044421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cf781-51f1-4b78-8feb-2ae4f9f8ea30 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:08.044700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cf781-51f1-4b78-8feb-2ae4f9f8ea30 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:08.044970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cf781-51f1-4b78-8feb-2ae4f9f8ea30 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:08.045592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cf781-51f1-4b78-8feb-2ae4f9f8ea30 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:18:08.045894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cf781-51f1-4b78-8feb-2ae4f9f8ea30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:08.047350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cf781-51f1-4b78-8feb-2ae4f9f8ea30 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:08.052267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 650/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Sep 21 05:18:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:18:08.058132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a3434-024f-49eb-b7a6-c517ebd98dab None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:08.089816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:08.155723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a3434-024f-49eb-b7a6-c517ebd98dab None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:08.187862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a3dfb-7d4b-4e9a-9bac-6f9a66f14cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:08.188455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a3dfb-7d4b-4e9a-9bac-6f9a66f14cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:08.188738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a3dfb-7d4b-4e9a-9bac-6f9a66f14cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:08.189012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a3dfb-7d4b-4e9a-9bac-6f9a66f14cc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:08.189699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a3dfb-7d4b-4e9a-9bac-6f9a66f14cc9 None admin] RBAC: Adding query filter params (interface=public, service_id=34f62d27f57b43d884b02ab0dcf29bf5, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:18:08.190034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a3dfb-7d4b-4e9a-9bac-6f9a66f14cc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:08.191378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a3dfb-7d4b-4e9a-9bac-6f9a66f14cc9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:08.196159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 650/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Sep 21 05:18:08 2022] GET /identity/v3/endpoints?service_id=34f62d27f57b43d884b02ab0dcf29bf5&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:18:08.201643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cf781-51f1-4b78-8feb-2ae4f9f8ea30 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:08.286182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:08.304656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cf781-51f1-4b78-8feb-2ae4f9f8ea30 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:08.338002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468ed4b-1ac4-405e-9214-93b410836d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:08.338606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468ed4b-1ac4-405e-9214-93b410836d3a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:08.338888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468ed4b-1ac4-405e-9214-93b410836d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:08.339164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468ed4b-1ac4-405e-9214-93b410836d3a None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:08.339760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468ed4b-1ac4-405e-9214-93b410836d3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:08.341115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468ed4b-1ac4-405e-9214-93b410836d3a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:08.347675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 651/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Sep 21 05:18:08 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:18:10.472830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a41344-f331-4d0b-8029-68eeb1b87b2a None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:10.473366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a41344-f331-4d0b-8029-68eeb1b87b2a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:10.473366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a41344-f331-4d0b-8029-68eeb1b87b2a None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:10.474752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 651/1302] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:18:10.480080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a4026-bd60-4c47-8d50-0f44cde47feb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:10.480501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a4026-bd60-4c47-8d50-0f44cde47feb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:10.480627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a4026-bd60-4c47-8d50-0f44cde47feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:10.500287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-163a4026-bd60-4c47-8d50-0f44cde47feb None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:18:10.501464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-163a4026-bd60-4c47-8d50-0f44cde47feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:18:10.567701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 652/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:18:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 05:18:10.841172 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a41344-f331-4d0b-8029-68eeb1b87b2a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:10.936585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:10.943649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a41344-f331-4d0b-8029-68eeb1b87b2a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:10.979045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a07e2a-2d72-46a4-87db-20a7f99b102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:10.979660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a07e2a-2d72-46a4-87db-20a7f99b102c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:10.979936 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a07e2a-2d72-46a4-87db-20a7f99b102c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:10.980201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a07e2a-2d72-46a4-87db-20a7f99b102c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:10.980763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a07e2a-2d72-46a4-87db-20a7f99b102c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:10.982145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a07e2a-2d72-46a4-87db-20a7f99b102c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-03a07e2a-2d72-46a4-87db-20a7f99b102c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:18:10.990889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:18:10.992061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:18:10.993061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 652/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Sep 21 05:18:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 05:18:10.998133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a4026-bd60-4c47-8d50-0f44cde47feb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:11.075475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:11.098544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a4026-bd60-4c47-8d50-0f44cde47feb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:11.131238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc3e81-573d-41d0-ac4e-a599fa7d25ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:11.131828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc3e81-573d-41d0-ac4e-a599fa7d25ac None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:11.132107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc3e81-573d-41d0-ac4e-a599fa7d25ac None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:11.132379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc3e81-573d-41d0-ac4e-a599fa7d25ac None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:11.132992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc3e81-573d-41d0-ac4e-a599fa7d25ac None admin] RBAC: Adding query filter params (name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:18:11.133295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc3e81-573d-41d0-ac4e-a599fa7d25ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:11.134688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc3e81-573d-41d0-ac4e-a599fa7d25ac None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:11.139593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 653/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Sep 21 05:18:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:18:11.145322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a07e2a-2d72-46a4-87db-20a7f99b102c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:18:11.241833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a07e2a-2d72-46a4-87db-20a7f99b102c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:18:11.273706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488b0de-6abf-4f6b-997c-85a9f84e46e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:18:11.274318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b0de-6abf-4f6b-997c-85a9f84e46e4 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:18:11.274597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b0de-6abf-4f6b-997c-85a9f84e46e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:18:11.274868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b0de-6abf-4f6b-997c-85a9f84e46e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:18:11.275526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b0de-6abf-4f6b-997c-85a9f84e46e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:18:11.276861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b0de-6abf-4f6b-997c-85a9f84e46e4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:18:11.295320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 653/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:18:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:18:21.136687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=30 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:18:21.291142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:15.615931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70845008-691b-4d9f-bf49-37457877cc75 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:15.616407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70845008-691b-4d9f-bf49-37457877cc75 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:15.616486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70845008-691b-4d9f-bf49-37457877cc75 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:15.617836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 654/1307] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:15.623311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942174c4-8484-4142-90e8-bc9ee9ecb9b3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:15.623724 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942174c4-8484-4142-90e8-bc9ee9ecb9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:15.623849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942174c4-8484-4142-90e8-bc9ee9ecb9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:15.777797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-942174c4-8484-4142-90e8-bc9ee9ecb9b3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:15.779509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-942174c4-8484-4142-90e8-bc9ee9ecb9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:15.930612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 654/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:16.215836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df2466-0471-4d98-b61b-5c43f1e160f7 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:16.216366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df2466-0471-4d98-b61b-5c43f1e160f7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:16.216822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df2466-0471-4d98-b61b-5c43f1e160f7 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:16.218721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 655/1309] 10.222.0.18 () {54 vars in 963 bytes} [Wed Sep 21 05:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:16.225972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcef9f-06f3-43cf-b2a3-b0328067fe72 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:16.226249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcef9f-06f3-43cf-b2a3-b0328067fe72 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:16.226608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcef9f-06f3-43cf-b2a3-b0328067fe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:16.311025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-40bcef9f-06f3-43cf-b2a3-b0328067fe72 None None] MFA Rules not processed for user `c07403facfaf4c71ab5324af6c44f52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:16.312185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-40bcef9f-06f3-43cf-b2a3-b0328067fe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:16.334162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:16.393786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 655/1310] 10.222.0.18 () {58 vars in 1060 bytes} [Wed Sep 21 05:19:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:16.403603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df2466-0471-4d98-b61b-5c43f1e160f7 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:16.405616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e7df2466-0471-4d98-b61b-5c43f1e160f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:16.522501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df2466-0471-4d98-b61b-5c43f1e160f7 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:16.603433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb54a8-177c-48e5-b278-684c02f1269e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:16.604251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb54a8-177c-48e5-b278-684c02f1269e service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:16.604707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb54a8-177c-48e5-b278-684c02f1269e service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:16.605134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb54a8-177c-48e5-b278-684c02f1269e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:16.607121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-46fb54a8-177c-48e5-b278-684c02f1269e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:16.670762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:16.740954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb54a8-177c-48e5-b278-684c02f1269e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:16.742471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb54a8-177c-48e5-b278-684c02f1269e service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:16.776442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 656/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:19.252615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79410e9a-f07e-4740-b8b5-c7faa880c219 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:19.253120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79410e9a-f07e-4740-b8b5-c7faa880c219 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:19.253120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79410e9a-f07e-4740-b8b5-c7faa880c219 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:19.254528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 656/1312] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:19.259785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3038b-6843-4c98-af05-0b433dc95ceb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:19.260180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3038b-6843-4c98-af05-0b433dc95ceb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:19.260322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3038b-6843-4c98-af05-0b433dc95ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:19.329369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-36a3038b-6843-4c98-af05-0b433dc95ceb None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:19.330592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-36a3038b-6843-4c98-af05-0b433dc95ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:19.424637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 657/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:19.703485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79410e9a-f07e-4740-b8b5-c7faa880c219 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:19.705098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-79410e9a-f07e-4740-b8b5-c7faa880c219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:19.810285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79410e9a-f07e-4740-b8b5-c7faa880c219 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:19.881145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:19.883524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a258fa-0c0a-4f12-872c-9c2a6016c21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:19.884333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a258fa-0c0a-4f12-872c-9c2a6016c21c service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:19.884625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a258fa-0c0a-4f12-872c-9c2a6016c21c service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:19.885062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a258fa-0c0a-4f12-872c-9c2a6016c21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:19.886902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-04a258fa-0c0a-4f12-872c-9c2a6016c21c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:19.994892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a258fa-0c0a-4f12-872c-9c2a6016c21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:19.996399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a258fa-0c0a-4f12-872c-9c2a6016c21c service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:20.014650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 657/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:22.499250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb6152-f0c7-42a9-bb4a-22916467fd48 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:22.499822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb6152-f0c7-42a9-bb4a-22916467fd48 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:22.499822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb6152-f0c7-42a9-bb4a-22916467fd48 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:22.501119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 658/1315] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:22.508799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ed8c0-308e-4290-882a-7cffbaa498bf None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:22.508799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ed8c0-308e-4290-882a-7cffbaa498bf None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:22.508799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ed8c0-308e-4290-882a-7cffbaa498bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:22.572542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-cb9ed8c0-308e-4290-882a-7cffbaa498bf None None] MFA Rules not processed for user `4a89a68d10724094982d478caa7d87f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:22.573742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cb9ed8c0-308e-4290-882a-7cffbaa498bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:22.624488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:22.669169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 658/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:22.968325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb6152-f0c7-42a9-bb4a-22916467fd48 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:22.969851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c6fb6152-f0c7-42a9-bb4a-22916467fd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:22.979521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:23.075748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb6152-f0c7-42a9-bb4a-22916467fd48 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:23.147361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b55025-769e-48c5-812e-ee6d0e0a10ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:23.147886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b55025-769e-48c5-812e-ee6d0e0a10ac service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:23.148164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b55025-769e-48c5-812e-ee6d0e0a10ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:23.148434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b55025-769e-48c5-812e-ee6d0e0a10ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:23.150047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-34b55025-769e-48c5-812e-ee6d0e0a10ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:23.210650 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:23.258011 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b55025-769e-48c5-812e-ee6d0e0a10ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:23.259453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b55025-769e-48c5-812e-ee6d0e0a10ac service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:23.276806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 659/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:25.931407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc06e0-4643-452a-b713-bd29a2d6eb10 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:25.931954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc06e0-4643-452a-b713-bd29a2d6eb10 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:25.931954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc06e0-4643-452a-b713-bd29a2d6eb10 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:25.933194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 659/1318] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:25.938324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc463262-9548-4dcf-8134-7bd4c9c39e1e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:25.938728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc463262-9548-4dcf-8134-7bd4c9c39e1e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:25.938882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc463262-9548-4dcf-8134-7bd4c9c39e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:26.003009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fc463262-9548-4dcf-8134-7bd4c9c39e1e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:26.004208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fc463262-9548-4dcf-8134-7bd4c9c39e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:26.097421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 660/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:26.398208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c92e31-e637-4d1e-8df3-1cfdc1b89aa6 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:26.398491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c92e31-e637-4d1e-8df3-1cfdc1b89aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:26.398770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c92e31-e637-4d1e-8df3-1cfdc1b89aa6 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:26.400098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 660/1320] 10.222.0.18 () {54 vars in 963 bytes} [Wed Sep 21 05:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:26.405766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3302f87-c759-485e-8457-80d6ba8a4ffa None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:26.406096 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3302f87-c759-485e-8457-80d6ba8a4ffa None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:26.406388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3302f87-c759-485e-8457-80d6ba8a4ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:26.429463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:26.481753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f3302f87-c759-485e-8457-80d6ba8a4ffa None None] MFA Rules not processed for user `c07403facfaf4c71ab5324af6c44f52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:26.482992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f3302f87-c759-485e-8457-80d6ba8a4ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:26.556550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 661/1321] 10.222.0.18 () {58 vars in 1060 bytes} [Wed Sep 21 05:19:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:26.565642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c92e31-e637-4d1e-8df3-1cfdc1b89aa6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:26.567302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-27c92e31-e637-4d1e-8df3-1cfdc1b89aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:26.671396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c92e31-e637-4d1e-8df3-1cfdc1b89aa6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:26.743476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c98dd-31f1-479e-b5f8-2347a246aea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:26.744081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c98dd-31f1-479e-b5f8-2347a246aea9 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:26.744360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c98dd-31f1-479e-b5f8-2347a246aea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:26.744632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c98dd-31f1-479e-b5f8-2347a246aea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:26.746254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f89c98dd-31f1-479e-b5f8-2347a246aea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:26.753001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:26.854013 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c98dd-31f1-479e-b5f8-2347a246aea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:26.855434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c98dd-31f1-479e-b5f8-2347a246aea9 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:26.873449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 661/1322] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:30.150947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499fe7-c99f-47a1-8537-478e6cc17c5e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:30.151484 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499fe7-c99f-47a1-8537-478e6cc17c5e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:30.151484 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499fe7-c99f-47a1-8537-478e6cc17c5e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:30.152849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 662/1323] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:30.158074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c19132-1d8a-4a42-a524-2173769faa54 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:30.158483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c19132-1d8a-4a42-a524-2173769faa54 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:30.158630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c19132-1d8a-4a42-a524-2173769faa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:30.233470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-57c19132-1d8a-4a42-a524-2173769faa54 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:30.234775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-57c19132-1d8a-4a42-a524-2173769faa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:30.296149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:30.337277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 662/1324] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:30.621610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499fe7-c99f-47a1-8537-478e6cc17c5e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:30.623186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ad499fe7-c99f-47a1-8537-478e6cc17c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:30.728013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499fe7-c99f-47a1-8537-478e6cc17c5e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:30.741631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:30.803064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1967a7-da61-4f59-9cac-c76b89f79fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:30.803669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1967a7-da61-4f59-9cac-c76b89f79fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:30.803946 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1967a7-da61-4f59-9cac-c76b89f79fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:30.804215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1967a7-da61-4f59-9cac-c76b89f79fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:30.805795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9a1967a7-da61-4f59-9cac-c76b89f79fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:30.914104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1967a7-da61-4f59-9cac-c76b89f79fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:30.915539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1967a7-da61-4f59-9cac-c76b89f79fc6 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:30.933532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 663/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:33.138865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdf4ab-f9f6-48c6-bec9-4cc5bba1911e None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:33.139440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdf4ab-f9f6-48c6-bec9-4cc5bba1911e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:33.139440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdf4ab-f9f6-48c6-bec9-4cc5bba1911e None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:33.140817 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 663/1326] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:33.146876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92e08e-31fe-42a9-afa9-3531a9f74642 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:33.147278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92e08e-31fe-42a9-afa9-3531a9f74642 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:33.147429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92e08e-31fe-42a9-afa9-3531a9f74642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:33.195217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:33.214466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-db92e08e-31fe-42a9-afa9-3531a9f74642 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:33.215683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-db92e08e-31fe-42a9-afa9-3531a9f74642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:33.321707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 664/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:33.602236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdf4ab-f9f6-48c6-bec9-4cc5bba1911e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:33.603790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7bcdf4ab-f9f6-48c6-bec9-4cc5bba1911e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:33.705883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdf4ab-f9f6-48c6-bec9-4cc5bba1911e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:33.782621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a3765-1334-4794-820c-7fa753af6821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:33.783221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a3765-1334-4794-820c-7fa753af6821 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:33.783496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a3765-1334-4794-820c-7fa753af6821 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:33.783767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a3765-1334-4794-820c-7fa753af6821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:33.785876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-179a3765-1334-4794-820c-7fa753af6821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:33.805559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:33.898317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a3765-1334-4794-820c-7fa753af6821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:33.899665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a3765-1334-4794-820c-7fa753af6821 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:33.918042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 664/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:36.133175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105099ac-319e-4833-9f60-a29f9449df53 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:36.133665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105099ac-319e-4833-9f60-a29f9449df53 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:36.133665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105099ac-319e-4833-9f60-a29f9449df53 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:36.135074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 665/1329] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:36.140391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f14015-5246-4cf9-bfe0-bc39bfa7d471 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:36.140816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f14015-5246-4cf9-bfe0-bc39bfa7d471 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:36.140946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f14015-5246-4cf9-bfe0-bc39bfa7d471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:36.206228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f0f14015-5246-4cf9-bfe0-bc39bfa7d471 None None] MFA Rules not processed for user `4a89a68d10724094982d478caa7d87f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:36.207915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f0f14015-5246-4cf9-bfe0-bc39bfa7d471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:36.274132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:36.307841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 665/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:36.590468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105099ac-319e-4833-9f60-a29f9449df53 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:36.592132 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-105099ac-319e-4833-9f60-a29f9449df53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:36.702540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105099ac-319e-4833-9f60-a29f9449df53 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:36.725563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:36.780034 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1a16e-6649-4d01-bd2a-3bd312625338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:36.780680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1a16e-6649-4d01-bd2a-3bd312625338 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:36.780971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1a16e-6649-4d01-bd2a-3bd312625338 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:36.781238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1a16e-6649-4d01-bd2a-3bd312625338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:36.782847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b0d1a16e-6649-4d01-bd2a-3bd312625338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:36.890299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1a16e-6649-4d01-bd2a-3bd312625338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:36.891758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1a16e-6649-4d01-bd2a-3bd312625338 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:36.911653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 666/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:39.349817 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05375623-ab05-492d-aa3d-6cb6e96de5bf None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:39.350385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05375623-ab05-492d-aa3d-6cb6e96de5bf None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:39.350497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05375623-ab05-492d-aa3d-6cb6e96de5bf None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:39.351764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 666/1332] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:39.357008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804cb6-5734-4dd0-86cf-f1cf32e8d370 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:39.357444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804cb6-5734-4dd0-86cf-f1cf32e8d370 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:39.357549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804cb6-5734-4dd0-86cf-f1cf32e8d370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:39.420509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:39.423139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-02804cb6-5734-4dd0-86cf-f1cf32e8d370 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:39.424848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-02804cb6-5734-4dd0-86cf-f1cf32e8d370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:39.519841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 667/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:39.803256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05375623-ab05-492d-aa3d-6cb6e96de5bf None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:39.804793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-05375623-ab05-492d-aa3d-6cb6e96de5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:39.905619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05375623-ab05-492d-aa3d-6cb6e96de5bf None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:39.978221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befc110-1cdb-4680-a288-11f5e81bb518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:39.978815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befc110-1cdb-4680-a288-11f5e81bb518 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:39.979090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befc110-1cdb-4680-a288-11f5e81bb518 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:39.979356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befc110-1cdb-4680-a288-11f5e81bb518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:39.980914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7befc110-1cdb-4680-a288-11f5e81bb518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:40.007566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:40.087899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befc110-1cdb-4680-a288-11f5e81bb518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:40.089322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befc110-1cdb-4680-a288-11f5e81bb518 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:40.107494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 667/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:42.291969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a49d-3877-40bf-9af0-5e9ac36e9987 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:42.292496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a49d-3877-40bf-9af0-5e9ac36e9987 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:42.292496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a49d-3877-40bf-9af0-5e9ac36e9987 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:42.293871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 668/1335] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:19:42.298967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d614f3d-d73e-4bf5-8ca1-644b83363597 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:42.299374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d614f3d-d73e-4bf5-8ca1-644b83363597 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:42.299528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d614f3d-d73e-4bf5-8ca1-644b83363597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:42.372987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5d614f3d-d73e-4bf5-8ca1-644b83363597 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:42.374230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5d614f3d-d73e-4bf5-8ca1-644b83363597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:42.449086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:42.470975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 668/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:42.756523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a49d-3877-40bf-9af0-5e9ac36e9987 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:42.758106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0eb6a49d-3877-40bf-9af0-5e9ac36e9987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:42.859005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a49d-3877-40bf-9af0-5e9ac36e9987 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:42.886767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:42.932443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069eea-3a12-4b6a-8fca-af11d3960e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:42.933032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069eea-3a12-4b6a-8fca-af11d3960e9f service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:42.933304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069eea-3a12-4b6a-8fca-af11d3960e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:42.933566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069eea-3a12-4b6a-8fca-af11d3960e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:42.935158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a4069eea-3a12-4b6a-8fca-af11d3960e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:43.039744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069eea-3a12-4b6a-8fca-af11d3960e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:43.041085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069eea-3a12-4b6a-8fca-af11d3960e9f service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:43.059216 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 669/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:45.587196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9751a8-b7d8-4ec6-966c-3b041987cdbd None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:45.587775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9751a8-b7d8-4ec6-966c-3b041987cdbd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:45.587775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9751a8-b7d8-4ec6-966c-3b041987cdbd None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:45.589086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 669/1338] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:45.594386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592aae1-78a0-429f-8aad-b9cae679f8e5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:45.594791 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592aae1-78a0-429f-8aad-b9cae679f8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:45.594930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592aae1-78a0-429f-8aad-b9cae679f8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:45.660127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2592aae1-78a0-429f-8aad-b9cae679f8e5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:45.661351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2592aae1-78a0-429f-8aad-b9cae679f8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:45.670905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:45.768167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 670/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:46.049502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9751a8-b7d8-4ec6-966c-3b041987cdbd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:46.051675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cf9751a8-b7d8-4ec6-966c-3b041987cdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:46.151394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9751a8-b7d8-4ec6-966c-3b041987cdbd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:46.223097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895368ae-d9a2-499e-a28a-dca1c48e82c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:46.223684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895368ae-d9a2-499e-a28a-dca1c48e82c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:46.223960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895368ae-d9a2-499e-a28a-dca1c48e82c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:46.224230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895368ae-d9a2-499e-a28a-dca1c48e82c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:46.225786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-895368ae-d9a2-499e-a28a-dca1c48e82c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:46.261509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:46.333629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895368ae-d9a2-499e-a28a-dca1c48e82c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:46.335086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895368ae-d9a2-499e-a28a-dca1c48e82c2 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:46.353385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 670/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:50.564920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066af40-bd36-4e92-9da8-d2b8d134bae0 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:50.565440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066af40-bd36-4e92-9da8-d2b8d134bae0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:50.565440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066af40-bd36-4e92-9da8-d2b8d134bae0 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:50.566794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 671/1341] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:50.572041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001d396-52e1-4f7b-bf3e-717eafe3053e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:50.572507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001d396-52e1-4f7b-bf3e-717eafe3053e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:50.572611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001d396-52e1-4f7b-bf3e-717eafe3053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:50.647267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f001d396-52e1-4f7b-bf3e-717eafe3053e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:50.648481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f001d396-52e1-4f7b-bf3e-717eafe3053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:50.743959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:50.755743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 671/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:51.044769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066af40-bd36-4e92-9da8-d2b8d134bae0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:51.046444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8066af40-bd36-4e92-9da8-d2b8d134bae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:51.149656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066af40-bd36-4e92-9da8-d2b8d134bae0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:51.188124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:51.225563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb29049-c7c5-423b-ae73-4d3513b049a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:51.226197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb29049-c7c5-423b-ae73-4d3513b049a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:51.226476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb29049-c7c5-423b-ae73-4d3513b049a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:51.226749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb29049-c7c5-423b-ae73-4d3513b049a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:51.228341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-efb29049-c7c5-423b-ae73-4d3513b049a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:51.339540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb29049-c7c5-423b-ae73-4d3513b049a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:51.341023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb29049-c7c5-423b-ae73-4d3513b049a4 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:51.358970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 672/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:54.107650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22d73e-ae83-4c58-814c-5407959fd494 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:54.108181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22d73e-ae83-4c58-814c-5407959fd494 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:54.108181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22d73e-ae83-4c58-814c-5407959fd494 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:54.109533 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 672/1344] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:54.115440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7598b2-8cbd-4da4-9ee7-1c355875e633 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:54.115878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7598b2-8cbd-4da4-9ee7-1c355875e633 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:54.116026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7598b2-8cbd-4da4-9ee7-1c355875e633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:54.181218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ad7598b2-8cbd-4da4-9ee7-1c355875e633 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:54.182464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ad7598b2-8cbd-4da4-9ee7-1c355875e633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:54.200385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:54.281145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 673/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:54.562204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22d73e-ae83-4c58-814c-5407959fd494 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:54.563748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ba22d73e-ae83-4c58-814c-5407959fd494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:54.673237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22d73e-ae83-4c58-814c-5407959fd494 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:54.747192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf714543-8926-4077-a92f-0b7d972a7505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:54.747798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf714543-8926-4077-a92f-0b7d972a7505 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:54.748079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf714543-8926-4077-a92f-0b7d972a7505 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:54.748352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf714543-8926-4077-a92f-0b7d972a7505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:54.749927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bf714543-8926-4077-a92f-0b7d972a7505 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:54.793414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:54.861686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf714543-8926-4077-a92f-0b7d972a7505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:54.863511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf714543-8926-4077-a92f-0b7d972a7505 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:54.883104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 673/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:19:59.120787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ee77a-c5bb-4636-9257-8dc67f4a7525 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:59.121288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ee77a-c5bb-4636-9257-8dc67f4a7525 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:59.121395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ee77a-c5bb-4636-9257-8dc67f4a7525 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:59.123892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 674/1347] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:19:59.129138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1dfe6-ab28-447b-a6b8-b804ae260ed7 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:59.130048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1dfe6-ab28-447b-a6b8-b804ae260ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:59.130689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1dfe6-ab28-447b-a6b8-b804ae260ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:59.207110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-bbb1dfe6-ab28-447b-a6b8-b804ae260ed7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:19:59.208362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bbb1dfe6-ab28-447b-a6b8-b804ae260ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:59.308480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:59.311002 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 674/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:19:59.594029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ee77a-c5bb-4636-9257-8dc67f4a7525 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:19:59.595577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-196ee77a-c5bb-4636-9257-8dc67f4a7525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:59.699369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ee77a-c5bb-4636-9257-8dc67f4a7525 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:19:59.757851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:19:59.784463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226d0d4-2040-414c-aed7-5f1e50751473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:19:59.785071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226d0d4-2040-414c-aed7-5f1e50751473 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:19:59.785342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226d0d4-2040-414c-aed7-5f1e50751473 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:19:59.785605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226d0d4-2040-414c-aed7-5f1e50751473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:19:59.787217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4226d0d4-2040-414c-aed7-5f1e50751473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:19:59.896197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226d0d4-2040-414c-aed7-5f1e50751473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:19:59.897556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226d0d4-2040-414c-aed7-5f1e50751473 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:19:59.915494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 675/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:04.778937 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae18df5-6bba-4cd0-aa1b-e460a46f7e34 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:04.779439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae18df5-6bba-4cd0-aa1b-e460a46f7e34 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:04.779439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae18df5-6bba-4cd0-aa1b-e460a46f7e34 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:04.780889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 675/1350] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:20:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:04.788020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ee83f-e637-4c7b-91df-a43bd138df28 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:04.788459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ee83f-e637-4c7b-91df-a43bd138df28 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:04.790470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ee83f-e637-4c7b-91df-a43bd138df28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:04.900769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-723ee83f-e637-4c7b-91df-a43bd138df28 None None] MFA Rules not processed for user `4b3d8a1c8b6640ae9e90ece0bbfbf410`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:04.902368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-723ee83f-e637-4c7b-91df-a43bd138df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:04.924868 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:05.057125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 676/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:20:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:09.221981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e89c4f-f3fa-4d81-864b-ab46d7b88bfb None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:09.222486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e89c4f-f3fa-4d81-864b-ab46d7b88bfb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:09.222486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e89c4f-f3fa-4d81-864b-ab46d7b88bfb None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:09.223771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 676/1352] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:09.228712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad608a-66c4-4026-a2cd-2e29f1149a00 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:09.229145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad608a-66c4-4026-a2cd-2e29f1149a00 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:09.229672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad608a-66c4-4026-a2cd-2e29f1149a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:09.317001 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5fad608a-66c4-4026-a2cd-2e29f1149a00 None None] MFA Rules not processed for user `4b3d8a1c8b6640ae9e90ece0bbfbf410`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:09.318423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5fad608a-66c4-4026-a2cd-2e29f1149a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:09.392681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:09.431323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 677/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:20:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:10.772283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704413e-4d5e-4fd7-a2ca-ca43cec6497c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:10.772815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704413e-4d5e-4fd7-a2ca-ca43cec6497c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:10.772930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704413e-4d5e-4fd7-a2ca-ca43cec6497c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:10.774551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 677/1354] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:10.779602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f63cf-5f9d-441a-88d9-c8e36dcbee1c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:10.780043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f63cf-5f9d-441a-88d9-c8e36dcbee1c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:10.780139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f63cf-5f9d-441a-88d9-c8e36dcbee1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:10.856988 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-192f63cf-5f9d-441a-88d9-c8e36dcbee1c None None] MFA Rules not processed for user `4b3d8a1c8b6640ae9e90ece0bbfbf410`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:10.858142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-192f63cf-5f9d-441a-88d9-c8e36dcbee1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:10.965002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 678/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:20:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:20:14.548751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef91b-1317-4b6f-b911-feb463375bdd None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:14.549606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef91b-1317-4b6f-b911-feb463375bdd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:14.550248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef91b-1317-4b6f-b911-feb463375bdd None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:14.555916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 678/1356] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:14.558398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb52cd1-df04-4928-bdb4-62b32e9a2ffe None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:14.559166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb52cd1-df04-4928-bdb4-62b32e9a2ffe None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:14.559439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb52cd1-df04-4928-bdb4-62b32e9a2ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:14.604093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:14.639532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fbb52cd1-df04-4928-bdb4-62b32e9a2ffe None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:14.640794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fbb52cd1-df04-4928-bdb4-62b32e9a2ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:14.750036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 679/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:20:15.051223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd23c3-66a8-456b-afd6-4755c11efe59 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:15.051942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd23c3-66a8-456b-afd6-4755c11efe59 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:15.052442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd23c3-66a8-456b-afd6-4755c11efe59 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:15.055258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 679/1358] 10.222.0.18 () {54 vars in 965 bytes} [Wed Sep 21 05:20:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:15.064513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14e902-c621-4bd2-95d0-07de5f7209ae None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:15.064866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14e902-c621-4bd2-95d0-07de5f7209ae None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:15.065219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14e902-c621-4bd2-95d0-07de5f7209ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:15.164689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9d14e902-c621-4bd2-95d0-07de5f7209ae None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:15.165899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9d14e902-c621-4bd2-95d0-07de5f7209ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:15.215444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:15.290650 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 680/1359] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Sep 21 05:20:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:20:15.298959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd23c3-66a8-456b-afd6-4755c11efe59 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:15.301159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7dcd23c3-66a8-456b-afd6-4755c11efe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:15.453827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd23c3-66a8-456b-afd6-4755c11efe59 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:15.535235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:15.577654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb19fb-1c62-4982-8ac6-dc5608d80c78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:15.578346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb19fb-1c62-4982-8ac6-dc5608d80c78 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:15.578578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb19fb-1c62-4982-8ac6-dc5608d80c78 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:15.578823 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb19fb-1c62-4982-8ac6-dc5608d80c78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:15.580433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1aeb19fb-1c62-4982-8ac6-dc5608d80c78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:15.688010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb19fb-1c62-4982-8ac6-dc5608d80c78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:15.691060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb19fb-1c62-4982-8ac6-dc5608d80c78 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:15.722419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 680/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:18.117006 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a49604-7025-447d-9b46-982550161845 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:18.117439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a49604-7025-447d-9b46-982550161845 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:18.117439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a49604-7025-447d-9b46-982550161845 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:18.124340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f04f71-ff06-456b-a21c-40c346e6b476 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:18.124914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f04f71-ff06-456b-a21c-40c346e6b476 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:18.124914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f04f71-ff06-456b-a21c-40c346e6b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:18.129789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 681/1361] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:18 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:18.196264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-78f04f71-ff06-456b-a21c-40c346e6b476 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:18.197472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-78f04f71-ff06-456b-a21c-40c346e6b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:18.237735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:18.303214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 681/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:18.599001 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a49604-7025-447d-9b46-982550161845 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:18.600705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b2a49604-7025-447d-9b46-982550161845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:18.779162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a49604-7025-447d-9b46-982550161845 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:18.791115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:18.916493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36062e7-9d9d-4fac-bafe-c52e98e8a155 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:18.917110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36062e7-9d9d-4fac-bafe-c52e98e8a155 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:18.917389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36062e7-9d9d-4fac-bafe-c52e98e8a155 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:18.917692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36062e7-9d9d-4fac-bafe-c52e98e8a155 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:18.919342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a36062e7-9d9d-4fac-bafe-c52e98e8a155 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:19.012906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36062e7-9d9d-4fac-bafe-c52e98e8a155 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:19.014455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36062e7-9d9d-4fac-bafe-c52e98e8a155 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:19.032339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 682/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:21.534885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1e9d6-beac-49df-bcdd-cfe104cee6a5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:21.535404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1e9d6-beac-49df-bcdd-cfe104cee6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:21.535404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1e9d6-beac-49df-bcdd-cfe104cee6a5 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:21.536835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 682/1364] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:21.541951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950e8b5-d135-4775-9de1-de460d178d4d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:21.542401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950e8b5-d135-4775-9de1-de460d178d4d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:21.542483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950e8b5-d135-4775-9de1-de460d178d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:21.558747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:21.625208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9950e8b5-d135-4775-9de1-de460d178d4d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:21.626460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9950e8b5-d135-4775-9de1-de460d178d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:21.733281 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 683/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:22.024343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087405e2-7432-4486-8825-cae1acda1cf0 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:22.024706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087405e2-7432-4486-8825-cae1acda1cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:22.025051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087405e2-7432-4486-8825-cae1acda1cf0 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:22.026615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 683/1366] 10.222.0.18 () {54 vars in 965 bytes} [Wed Sep 21 05:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:22.032167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7bd07-3f57-459e-9063-79e0ebaf9406 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:22.032527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7bd07-3f57-459e-9063-79e0ebaf9406 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:22.032855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7bd07-3f57-459e-9063-79e0ebaf9406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:22.110352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b0a7bd07-3f57-459e-9063-79e0ebaf9406 None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:22.112350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b0a7bd07-3f57-459e-9063-79e0ebaf9406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:22.147891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:22.233318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 684/1367] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Sep 21 05:20:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:22.241605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087405e2-7432-4486-8825-cae1acda1cf0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:22.243332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-087405e2-7432-4486-8825-cae1acda1cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:22.398477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087405e2-7432-4486-8825-cae1acda1cf0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:22.420439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:22.520507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09484b-cd11-4928-b761-4b5621388e53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:22.521125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09484b-cd11-4928-b761-4b5621388e53 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:22.521405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09484b-cd11-4928-b761-4b5621388e53 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:22.521681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09484b-cd11-4928-b761-4b5621388e53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:22.523512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4c09484b-cd11-4928-b761-4b5621388e53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:22.628334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09484b-cd11-4928-b761-4b5621388e53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:22.630005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09484b-cd11-4928-b761-4b5621388e53 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:22.652565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 684/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:20:22.753116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0efa4-4700-4402-99fa-2524bbe6835b None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:22.753566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0efa4-4700-4402-99fa-2524bbe6835b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:22.753566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0efa4-4700-4402-99fa-2524bbe6835b None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:22.759370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 685/1369] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:22.765285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbc7fc-b285-4383-b3cc-af486600281d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:22.765580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbc7fc-b285-4383-b3cc-af486600281d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:22.765864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbc7fc-b285-4383-b3cc-af486600281d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:22.803237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:22.854289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-febbc7fc-b285-4383-b3cc-af486600281d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:22.855563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-febbc7fc-b285-4383-b3cc-af486600281d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:22.974079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 685/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:23.323295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fccec7-6ce1-475a-951a-3a9113bed94d None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:23.323712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fccec7-6ce1-475a-951a-3a9113bed94d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:23.324056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fccec7-6ce1-475a-951a-3a9113bed94d None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:23.325625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 686/1371] 10.222.0.18 () {54 vars in 962 bytes} [Wed Sep 21 05:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:23.330530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbababd-5709-4838-9e63-d8376633d79b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:23.330826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbababd-5709-4838-9e63-d8376633d79b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:23.331105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbababd-5709-4838-9e63-d8376633d79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:23.413449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-dbbababd-5709-4838-9e63-d8376633d79b None None] MFA Rules not processed for user `63a04998aafd40cc941283700898c92f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:23.414825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dbbababd-5709-4838-9e63-d8376633d79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:23.471437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:23.505793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 686/1372] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Sep 21 05:20:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:23.513258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fccec7-6ce1-475a-951a-3a9113bed94d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:23.514838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-89fccec7-6ce1-475a-951a-3a9113bed94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:23.631207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fccec7-6ce1-475a-951a-3a9113bed94d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:23.679784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:23.720401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f1deb-4249-4915-b20c-742a0d102de4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:23.721786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f1deb-4249-4915-b20c-742a0d102de4 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:23.721913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f1deb-4249-4915-b20c-742a0d102de4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:23.722128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f1deb-4249-4915-b20c-742a0d102de4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:23.723753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-947f1deb-4249-4915-b20c-742a0d102de4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:23.850404 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f1deb-4249-4915-b20c-742a0d102de4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:23.851905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f1deb-4249-4915-b20c-742a0d102de4 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:23.874230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 687/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:24.902494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c0ae6-a9f9-4976-92bc-b6f308ea23d4 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:24.903078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c0ae6-a9f9-4976-92bc-b6f308ea23d4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:24.903078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c0ae6-a9f9-4976-92bc-b6f308ea23d4 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:24.904380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 687/1374] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:24.910392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ed2c7-9957-460e-bca4-2c8421956532 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:24.910908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ed2c7-9957-460e-bca4-2c8421956532 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:24.911369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ed2c7-9957-460e-bca4-2c8421956532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:24.984248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ed9ed2c7-9957-460e-bca4-2c8421956532 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:24.985470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ed9ed2c7-9957-460e-bca4-2c8421956532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:25.007055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:25.083474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 688/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:20:25.383515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c0ae6-a9f9-4976-92bc-b6f308ea23d4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:25.385066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-198c0ae6-a9f9-4976-92bc-b6f308ea23d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:25.546128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c0ae6-a9f9-4976-92bc-b6f308ea23d4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:25.601901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:25.668981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5646e-65aa-4adf-bb36-d787fb6c5426 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:25.669593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5646e-65aa-4adf-bb36-d787fb6c5426 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:25.669874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5646e-65aa-4adf-bb36-d787fb6c5426 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:25.670182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5646e-65aa-4adf-bb36-d787fb6c5426 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:25.671795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-81c5646e-65aa-4adf-bb36-d787fb6c5426 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:25.766272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5646e-65aa-4adf-bb36-d787fb6c5426 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:25.768445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5646e-65aa-4adf-bb36-d787fb6c5426 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:25.788356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 688/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:28.000790 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d5465-7ca5-4322-a031-c2505779ef4e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:28.001312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d5465-7ca5-4322-a031-c2505779ef4e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:28.001312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d5465-7ca5-4322-a031-c2505779ef4e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:28.003579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 689/1377] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:28.008787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdde186-d0dc-4bde-a778-dc1221ee2d56 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:28.009211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdde186-d0dc-4bde-a778-dc1221ee2d56 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:28.009376 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdde186-d0dc-4bde-a778-dc1221ee2d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:28.076697 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-dbdde186-d0dc-4bde-a778-dc1221ee2d56 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:28.077901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dbdde186-d0dc-4bde-a778-dc1221ee2d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:28.085872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:28.188971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 689/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:28.473910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d5465-7ca5-4322-a031-c2505779ef4e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:28.475554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9d7d5465-7ca5-4322-a031-c2505779ef4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:28.642978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d5465-7ca5-4322-a031-c2505779ef4e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:28.646903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:28.759611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb82e8-1816-4c7f-be16-44b9fd12691d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:28.760122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb82e8-1816-4c7f-be16-44b9fd12691d service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:28.760396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb82e8-1816-4c7f-be16-44b9fd12691d service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:28.760663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb82e8-1816-4c7f-be16-44b9fd12691d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:28.762257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c1eb82e8-1816-4c7f-be16-44b9fd12691d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:28.854666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb82e8-1816-4c7f-be16-44b9fd12691d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:28.856245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb82e8-1816-4c7f-be16-44b9fd12691d service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:28.876592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 690/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:31.154056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77a8b-7736-45f4-8289-6c23d9358af8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:31.154555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77a8b-7736-45f4-8289-6c23d9358af8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:31.154664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77a8b-7736-45f4-8289-6c23d9358af8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:31.158654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 690/1380] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:31.161655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c009d-010b-4118-a1c2-2331df4dec82 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:31.162116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c009d-010b-4118-a1c2-2331df4dec82 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:31.162248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c009d-010b-4118-a1c2-2331df4dec82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:31.174567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:31.241266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fb1c009d-010b-4118-a1c2-2331df4dec82 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:31.242556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fb1c009d-010b-4118-a1c2-2331df4dec82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:31.350635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 691/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:31.687601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d77a8b-7736-45f4-8289-6c23d9358af8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:31.690246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f5d77a8b-7736-45f4-8289-6c23d9358af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:31.834742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:31.846162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d77a8b-7736-45f4-8289-6c23d9358af8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:31.962915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e59555-4548-4cdd-b770-5f8e37904f34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:31.963524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e59555-4548-4cdd-b770-5f8e37904f34 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:31.963805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e59555-4548-4cdd-b770-5f8e37904f34 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:31.964123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e59555-4548-4cdd-b770-5f8e37904f34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:31.965731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e6e59555-4548-4cdd-b770-5f8e37904f34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:32.067794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e59555-4548-4cdd-b770-5f8e37904f34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:32.069418 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e59555-4548-4cdd-b770-5f8e37904f34 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:32.079916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:32.089026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 691/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:34.585391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e2a0c-e6bd-4353-ace2-6b558e63170c None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:34.585919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e2a0c-e6bd-4353-ace2-6b558e63170c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:34.586029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e2a0c-e6bd-4353-ace2-6b558e63170c None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:34.588999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 692/1383] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:34.594297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0457f-c8c2-4e7a-9191-57c9f8b60442 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:34.594932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0457f-c8c2-4e7a-9191-57c9f8b60442 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:34.594932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0457f-c8c2-4e7a-9191-57c9f8b60442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:34.675264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-aad0457f-c8c2-4e7a-9191-57c9f8b60442 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:34.676532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aad0457f-c8c2-4e7a-9191-57c9f8b60442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:34.780706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 692/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:35.072856 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e2a0c-e6bd-4353-ace2-6b558e63170c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:35.074744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dd1e2a0c-e6bd-4353-ace2-6b558e63170c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:35.170430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:35.241328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e2a0c-e6bd-4353-ace2-6b558e63170c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:35.366550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5959c6-b4ff-4140-b626-bc4ee14b1c58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:35.367159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5959c6-b4ff-4140-b626-bc4ee14b1c58 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:35.367435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5959c6-b4ff-4140-b626-bc4ee14b1c58 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:35.367706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5959c6-b4ff-4140-b626-bc4ee14b1c58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:35.369310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7a5959c6-b4ff-4140-b626-bc4ee14b1c58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:35.422756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:35.470035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5959c6-b4ff-4140-b626-bc4ee14b1c58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:35.472727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5959c6-b4ff-4140-b626-bc4ee14b1c58 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:35.496849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 693/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:37.673602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870d3aa-7626-4a75-ac51-f7d4d9ad8375 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:37.674176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870d3aa-7626-4a75-ac51-f7d4d9ad8375 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:37.674176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870d3aa-7626-4a75-ac51-f7d4d9ad8375 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:37.675498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 693/1386] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:37.680793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb173a0d-8f39-452f-9c53-7fc6fd93df6b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:37.681244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb173a0d-8f39-452f-9c53-7fc6fd93df6b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:37.681342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb173a0d-8f39-452f-9c53-7fc6fd93df6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:37.747544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fb173a0d-8f39-452f-9c53-7fc6fd93df6b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:37.748749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fb173a0d-8f39-452f-9c53-7fc6fd93df6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:37.846630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 694/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:38.136796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870d3aa-7626-4a75-ac51-f7d4d9ad8375 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:38.138401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8870d3aa-7626-4a75-ac51-f7d4d9ad8375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:38.215825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:38.285452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870d3aa-7626-4a75-ac51-f7d4d9ad8375 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:38.397614 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726df6c-3214-4c43-b5e6-2b90af4415b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:38.398170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726df6c-3214-4c43-b5e6-2b90af4415b8 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:38.398447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726df6c-3214-4c43-b5e6-2b90af4415b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:38.398720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726df6c-3214-4c43-b5e6-2b90af4415b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:38.400303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1726df6c-3214-4c43-b5e6-2b90af4415b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:38.438977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:38.491640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726df6c-3214-4c43-b5e6-2b90af4415b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:38.493238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726df6c-3214-4c43-b5e6-2b90af4415b8 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:38.511520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 694/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:39.095758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda5330-b8cb-4ff3-886d-51137c0ab187 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:39.096310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda5330-b8cb-4ff3-886d-51137c0ab187 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:39.096310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda5330-b8cb-4ff3-886d-51137c0ab187 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:39.097757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 695/1389] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:39.103365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e65ab-a725-4dc1-b93c-520d6024b064 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:39.103797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e65ab-a725-4dc1-b93c-520d6024b064 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:39.103944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e65ab-a725-4dc1-b93c-520d6024b064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:39.169865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b81e65ab-a725-4dc1-b93c-520d6024b064 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:39.171107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b81e65ab-a725-4dc1-b93c-520d6024b064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:39.268509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 695/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:20:39.577252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d47d9-9173-492c-b80a-aa5126799275 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:39.577534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d47d9-9173-492c-b80a-aa5126799275 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:39.577808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d47d9-9173-492c-b80a-aa5126799275 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:39.579207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 696/1391] 10.222.0.18 () {54 vars in 962 bytes} [Wed Sep 21 05:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:39.584774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761c8c1-0251-4e4b-86ff-1e19fbcdbe87 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:39.585318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761c8c1-0251-4e4b-86ff-1e19fbcdbe87 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:39.585734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761c8c1-0251-4e4b-86ff-1e19fbcdbe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:39.601784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:39.665163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6761c8c1-0251-4e4b-86ff-1e19fbcdbe87 None None] MFA Rules not processed for user `5305f8a5d2b540de91d3caedad454bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:39.666413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6761c8c1-0251-4e4b-86ff-1e19fbcdbe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:39.743561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 696/1392] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Sep 21 05:20:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:39.751945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d47d9-9173-492c-b80a-aa5126799275 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:39.753621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0c3d47d9-9173-492c-b80a-aa5126799275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:39.784573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:39.857338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d47d9-9173-492c-b80a-aa5126799275 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:39.932141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a665279-c5fc-452d-9d04-f9573ffada5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5305f8a5d2b540de91d3caedad454bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:39.932739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a665279-c5fc-452d-9d04-f9573ffada5a service glance] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:39.933014 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a665279-c5fc-452d-9d04-f9573ffada5a service glance] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:39.933282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a665279-c5fc-452d-9d04-f9573ffada5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:39.934873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3a665279-c5fc-452d-9d04-f9573ffada5a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:40.014844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:40.047299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a665279-c5fc-452d-9d04-f9573ffada5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:40.049340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a665279-c5fc-452d-9d04-f9573ffada5a service glance] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:40.067488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 697/1393] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:40.731006 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc9c92-3617-4bdc-b9f7-9718bd922abb None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:40.732783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc9c92-3617-4bdc-b9f7-9718bd922abb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:40.732783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc9c92-3617-4bdc-b9f7-9718bd922abb None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:40.733421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 697/1394] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:40.739602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc38f15-7be8-4a80-8b48-fd21705abd47 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:40.742073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc38f15-7be8-4a80-8b48-fd21705abd47 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:40.742377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc38f15-7be8-4a80-8b48-fd21705abd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:40.814872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29415ab9-82b2-44a1-88ca-a9152882340f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:40.815209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29415ab9-82b2-44a1-88ca-a9152882340f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:40.815489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29415ab9-82b2-44a1-88ca-a9152882340f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:40.815930 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-adc38f15-7be8-4a80-8b48-fd21705abd47 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:40.817146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-adc38f15-7be8-4a80-8b48-fd21705abd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:40.885418 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-29415ab9-82b2-44a1-88ca-a9152882340f None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:40.886672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-29415ab9-82b2-44a1-88ca-a9152882340f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:40.915788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 698/1395] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:40.944594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:40.976804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 698/1396] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:20:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:20:41.020156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1106-9e3d-4130-9996-2a5733682243 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:41.020475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1106-9e3d-4130-9996-2a5733682243 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:41.020750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1106-9e3d-4130-9996-2a5733682243 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:41.022145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 699/1397] 10.222.0.18 () {54 vars in 943 bytes} [Wed Sep 21 05:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:41.028292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f347d4-4bfd-451f-a998-91fc119a0134 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:41.028641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f347d4-4bfd-451f-a998-91fc119a0134 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:41.028980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f347d4-4bfd-451f-a998-91fc119a0134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:41.105654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e0f347d4-4bfd-451f-a998-91fc119a0134 None None] MFA Rules not processed for user `ee0fe35d9e304ee495998c93c6909fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:41.106906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e0f347d4-4bfd-451f-a998-91fc119a0134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:41.204563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed1106-9e3d-4130-9996-2a5733682243 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:41.205887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-31ed1106-9e3d-4130-9996-2a5733682243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:41.217459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 699/1398] 10.222.0.18 () {58 vars in 1040 bytes} [Wed Sep 21 05:20:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:41.226879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f347d4-4bfd-451f-a998-91fc119a0134 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:41.237603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f347d4-4bfd-451f-a998-91fc119a0134 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:41.255389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:41.265244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:41.354670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d7125-6b92-4d92-8e08-d2497483ffad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:41.355565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7125-6b92-4d92-8e08-d2497483ffad service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:41.355898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7125-6b92-4d92-8e08-d2497483ffad service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:41.356118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7125-6b92-4d92-8e08-d2497483ffad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:41.357766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-300d7125-6b92-4d92-8e08-d2497483ffad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:41.377288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed1106-9e3d-4130-9996-2a5733682243 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:41.446832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d7125-6b92-4d92-8e08-d2497483ffad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:41.450004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d7125-6b92-4d92-8e08-d2497483ffad service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:41.456420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 700/1399] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:20:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 05:20:41.502294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:41.504808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba78da-7520-45b8-aab9-c48554f12410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:41.505608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba78da-7520-45b8-aab9-c48554f12410 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:41.506095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba78da-7520-45b8-aab9-c48554f12410 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:41.506775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba78da-7520-45b8-aab9-c48554f12410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:41.508497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f4ba78da-7520-45b8-aab9-c48554f12410 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:41.613286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba78da-7520-45b8-aab9-c48554f12410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:41.614989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba78da-7520-45b8-aab9-c48554f12410 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:41.633712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 700/1400] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:43.825009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42b259-f8ee-42c4-9b50-1bc0b208eb20 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:43.825577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42b259-f8ee-42c4-9b50-1bc0b208eb20 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:43.825577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42b259-f8ee-42c4-9b50-1bc0b208eb20 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:43.826914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 701/1401] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:43.834939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205605d-5624-46ce-a503-7a3f5077b2b4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:43.835458 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205605d-5624-46ce-a503-7a3f5077b2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:43.835540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205605d-5624-46ce-a503-7a3f5077b2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:43.906149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b205605d-5624-46ce-a503-7a3f5077b2b4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:43.907338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b205605d-5624-46ce-a503-7a3f5077b2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:43.983536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:44.014793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 701/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:44.359963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42b259-f8ee-42c4-9b50-1bc0b208eb20 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:44.361793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7a42b259-f8ee-42c4-9b50-1bc0b208eb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:44.432996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:44.529086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42b259-f8ee-42c4-9b50-1bc0b208eb20 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:44.666303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5158-1c03-4f40-b22f-dc3ff4bb4229 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:44.666904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5158-1c03-4f40-b22f-dc3ff4bb4229 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:44.667188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5158-1c03-4f40-b22f-dc3ff4bb4229 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:44.667467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5158-1c03-4f40-b22f-dc3ff4bb4229 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:44.669068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-91cf5158-1c03-4f40-b22f-dc3ff4bb4229 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:44.690131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:44.771624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5158-1c03-4f40-b22f-dc3ff4bb4229 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:44.773518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5158-1c03-4f40-b22f-dc3ff4bb4229 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:44.791929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 702/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:47.010165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31939-7b2c-4236-95e3-5a24acaa9982 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:47.010737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31939-7b2c-4236-95e3-5a24acaa9982 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:47.010737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31939-7b2c-4236-95e3-5a24acaa9982 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:47.015878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 702/1404] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:47.020567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091bb5-939a-4ec9-8e57-d6be674a30e9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:47.020567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091bb5-939a-4ec9-8e57-d6be674a30e9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:47.020567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091bb5-939a-4ec9-8e57-d6be674a30e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:47.089580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e3091bb5-939a-4ec9-8e57-d6be674a30e9 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:47.090873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e3091bb5-939a-4ec9-8e57-d6be674a30e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:47.196645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:47.205548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 703/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:20:47.498766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e31939-7b2c-4236-95e3-5a24acaa9982 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:47.500647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-05e31939-7b2c-4236-95e3-5a24acaa9982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:47.653485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e31939-7b2c-4236-95e3-5a24acaa9982 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:47.714253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:47.779613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6747ad-3629-4d48-b862-6bd42b0fd9e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:47.780563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6747ad-3629-4d48-b862-6bd42b0fd9e8 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:47.780844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6747ad-3629-4d48-b862-6bd42b0fd9e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:47.781116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6747ad-3629-4d48-b862-6bd42b0fd9e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:47.782786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8f6747ad-3629-4d48-b862-6bd42b0fd9e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:47.884010 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6747ad-3629-4d48-b862-6bd42b0fd9e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:47.885600 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6747ad-3629-4d48-b862-6bd42b0fd9e8 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:47.914114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 703/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:20:50.115751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bdde5-74cc-460b-9201-c5be84a44adc None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:50.116322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bdde5-74cc-460b-9201-c5be84a44adc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:50.116322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bdde5-74cc-460b-9201-c5be84a44adc None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:50.117794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 704/1407] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:20:50.123287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a28487-bf37-4943-ba00-288fa8e3c683 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:50.123719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a28487-bf37-4943-ba00-288fa8e3c683 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:50.123828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a28487-bf37-4943-ba00-288fa8e3c683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:50.194097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-22a28487-bf37-4943-ba00-288fa8e3c683 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:20:50.195317 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-22a28487-bf37-4943-ba00-288fa8e3c683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:50.210088 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:50.304883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 704/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:20:50.606058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bdde5-74cc-460b-9201-c5be84a44adc None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:20:50.607786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7c6bdde5-74cc-460b-9201-c5be84a44adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:50.778459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bdde5-74cc-460b-9201-c5be84a44adc None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:20:50.858007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:20:50.910383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf42eb-66d0-4457-8952-761929648ef0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:20:50.910994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf42eb-66d0-4457-8952-761929648ef0 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:20:50.911278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf42eb-66d0-4457-8952-761929648ef0 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:20:50.911546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf42eb-66d0-4457-8952-761929648ef0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:20:50.913165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-82bf42eb-66d0-4457-8952-761929648ef0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:20:51.008245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf42eb-66d0-4457-8952-761929648ef0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:20:51.009839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf42eb-66d0-4457-8952-761929648ef0 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:20:51.028798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 705/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:00.303442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=38 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:01.027087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=64 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:04.487835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf0286-fb79-4c9d-8126-f689e698833d None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:04.488412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf0286-fb79-4c9d-8126-f689e698833d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:04.488412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf0286-fb79-4c9d-8126-f689e698833d None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:04.489761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 705/1410] 10.222.0.18 () {54 vars in 918 bytes} [Wed Sep 21 05:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:04.495072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bc51c-d703-4f8f-9ada-2fe1f2475372 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:04.495518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bc51c-d703-4f8f-9ada-2fe1f2475372 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:04.495684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bc51c-d703-4f8f-9ada-2fe1f2475372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:04.572229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-601bc51c-d703-4f8f-9ada-2fe1f2475372 None None] MFA Rules not processed for user `4b3d8a1c8b6640ae9e90ece0bbfbf410`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:04.573436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-601bc51c-d703-4f8f-9ada-2fe1f2475372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:04.679173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 706/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Sep 21 05:21:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:04.693498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c9c39-65e2-41b6-81b5-8a1f1d774c21 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:04.693991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c9c39-65e2-41b6-81b5-8a1f1d774c21 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:04.694313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c9c39-65e2-41b6-81b5-8a1f1d774c21 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:04.696052 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 706/1412] 10.222.0.18 () {54 vars in 918 bytes} [Wed Sep 21 05:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:04.700694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348003b-1f89-4b64-9beb-6b11ecb436ae None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:04.701009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348003b-1f89-4b64-9beb-6b11ecb436ae None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:04.701285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348003b-1f89-4b64-9beb-6b11ecb436ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:04.773791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:04.776331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3348003b-1f89-4b64-9beb-6b11ecb436ae None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:04.778147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3348003b-1f89-4b64-9beb-6b11ecb436ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:04.892820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 707/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Sep 21 05:21:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:07.468995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d95280-a444-4062-a1d3-b96c7ea823fe None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:07.469533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d95280-a444-4062-a1d3-b96c7ea823fe None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:07.469664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d95280-a444-4062-a1d3-b96c7ea823fe None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:07.471552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:07.477010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fd9ab-df19-485e-aece-87c139e52101 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:07.477431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fd9ab-df19-485e-aece-87c139e52101 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:07.477568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fd9ab-df19-485e-aece-87c139e52101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:07.543514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-bf0fd9ab-df19-485e-aece-87c139e52101 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:07.544728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bf0fd9ab-df19-485e-aece-87c139e52101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:07.599605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:07.644144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:07.932377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d95280-a444-4062-a1d3-b96c7ea823fe None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:07.933892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-35d95280-a444-4062-a1d3-b96c7ea823fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:08.077333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d95280-a444-4062-a1d3-b96c7ea823fe None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:08.151361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:08.191077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c3970-bcc9-45e5-a0f8-7c9bc7953b60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:08.191609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c3970-bcc9-45e5-a0f8-7c9bc7953b60 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:08.191884 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c3970-bcc9-45e5-a0f8-7c9bc7953b60 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:08.192153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c3970-bcc9-45e5-a0f8-7c9bc7953b60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:08.193747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-746c3970-bcc9-45e5-a0f8-7c9bc7953b60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:08.282945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c3970-bcc9-45e5-a0f8-7c9bc7953b60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:08.284535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c3970-bcc9-45e5-a0f8-7c9bc7953b60 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:08.302523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 708/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:09.442926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fa2ca-dc75-4469-97aa-23b40b0abd72 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:09.443499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fa2ca-dc75-4469-97aa-23b40b0abd72 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:09.443499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fa2ca-dc75-4469-97aa-23b40b0abd72 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:09.444812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:09.450858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a04b9-15e2-48c9-a7ff-5c74e5dd51d3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:09.451255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a04b9-15e2-48c9-a7ff-5c74e5dd51d3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:09.451436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a04b9-15e2-48c9-a7ff-5c74e5dd51d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:09.526235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-260a04b9-15e2-48c9-a7ff-5c74e5dd51d3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:09.527447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-260a04b9-15e2-48c9-a7ff-5c74e5dd51d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:09.554481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:09.625185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:09.777705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fa2ca-dc75-4469-97aa-23b40b0abd72 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:09.779312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c54fa2ca-dc75-4469-97aa-23b40b0abd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:09.924380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fa2ca-dc75-4469-97aa-23b40b0abd72 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:09.963476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:10.036244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0fdc-8e6a-40b3-9c2b-32452ada237a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:10.036780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0fdc-8e6a-40b3-9c2b-32452ada237a service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:10.037061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0fdc-8e6a-40b3-9c2b-32452ada237a service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:10.037335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0fdc-8e6a-40b3-9c2b-32452ada237a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:10.038954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6e3a0fdc-8e6a-40b3-9c2b-32452ada237a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:10.139466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0fdc-8e6a-40b3-9c2b-32452ada237a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:10.141177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0fdc-8e6a-40b3-9c2b-32452ada237a service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:10.160037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 710/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:11.284078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aabf0-7ede-469e-b19e-d38476d29e1a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:11.284590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aabf0-7ede-469e-b19e-d38476d29e1a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:11.284590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aabf0-7ede-469e-b19e-d38476d29e1a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:11.285882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:11.292004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f514c9-9419-4ef0-a292-43c51c3e169d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:11.292456 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f514c9-9419-4ef0-a292-43c51c3e169d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:11.292561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f514c9-9419-4ef0-a292-43c51c3e169d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:11.348641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:11.370388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a3f514c9-9419-4ef0-a292-43c51c3e169d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:11.371599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a3f514c9-9419-4ef0-a292-43c51c3e169d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:11.467326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:11.617323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aabf0-7ede-469e-b19e-d38476d29e1a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:11.618898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d52aabf0-7ede-469e-b19e-d38476d29e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:11.764408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aabf0-7ede-469e-b19e-d38476d29e1a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:11.776578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:11.877770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd789fb-01e9-48b1-9d93-76d32bd0f593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:11.878328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd789fb-01e9-48b1-9d93-76d32bd0f593 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:11.878611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd789fb-01e9-48b1-9d93-76d32bd0f593 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:11.878886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd789fb-01e9-48b1-9d93-76d32bd0f593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:11.880488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8bd789fb-01e9-48b1-9d93-76d32bd0f593 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:11.968772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd789fb-01e9-48b1-9d93-76d32bd0f593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:11.970402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd789fb-01e9-48b1-9d93-76d32bd0f593 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:11.988613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 711/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:13.100002 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6539c-be17-4e68-bcc4-ca5f7348d6e6 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:13.100537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6539c-be17-4e68-bcc4-ca5f7348d6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:13.100537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6539c-be17-4e68-bcc4-ca5f7348d6e6 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:13.101869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:13.108191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf7fba-7963-470c-b97c-70ad43958803 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:13.108599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf7fba-7963-470c-b97c-70ad43958803 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:13.108748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf7fba-7963-470c-b97c-70ad43958803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:13.129199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:13.184416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fbbf7fba-7963-470c-b97c-70ad43958803 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:13.185626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fbbf7fba-7963-470c-b97c-70ad43958803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:13.280567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:13.432820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6539c-be17-4e68-bcc4-ca5f7348d6e6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:13.434345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-37b6539c-be17-4e68-bcc4-ca5f7348d6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:13.543338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:13.578737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6539c-be17-4e68-bcc4-ca5f7348d6e6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:13.690771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855106fe-71d8-4af9-9693-b8423f9cd4cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:13.691376 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855106fe-71d8-4af9-9693-b8423f9cd4cd service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:13.691656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855106fe-71d8-4af9-9693-b8423f9cd4cd service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:13.691931 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855106fe-71d8-4af9-9693-b8423f9cd4cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:13.693517 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-855106fe-71d8-4af9-9693-b8423f9cd4cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:13.772060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:13.786959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855106fe-71d8-4af9-9693-b8423f9cd4cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:13.788532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855106fe-71d8-4af9-9693-b8423f9cd4cd service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:13.807201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 713/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:14.904677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1c6b7-a602-487e-9914-a3889ecc3e46 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:14.905321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1c6b7-a602-487e-9914-a3889ecc3e46 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:14.905321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1c6b7-a602-487e-9914-a3889ecc3e46 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:14.906629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:14.912903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84fbfa-474d-4126-93ab-9cb2960400e4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:14.913361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84fbfa-474d-4126-93ab-9cb2960400e4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:14.913472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84fbfa-474d-4126-93ab-9cb2960400e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:14.993810 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4c84fbfa-474d-4126-93ab-9cb2960400e4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:14.995040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4c84fbfa-474d-4126-93ab-9cb2960400e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:15.087872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:15.242405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1c6b7-a602-487e-9914-a3889ecc3e46 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:15.243852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3dd1c6b7-a602-487e-9914-a3889ecc3e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:15.329364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:15.390671 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1c6b7-a602-487e-9914-a3889ecc3e46 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:15.514952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df94d3-86f8-4ad4-b392-2f2bcf7ba2fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:15.515883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df94d3-86f8-4ad4-b392-2f2bcf7ba2fb service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:15.516387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df94d3-86f8-4ad4-b392-2f2bcf7ba2fb service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:15.517401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df94d3-86f8-4ad4-b392-2f2bcf7ba2fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:15.520460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-93df94d3-86f8-4ad4-b392-2f2bcf7ba2fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:15.574209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:15.618543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df94d3-86f8-4ad4-b392-2f2bcf7ba2fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:15.620159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df94d3-86f8-4ad4-b392-2f2bcf7ba2fb service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:15.638726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 714/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:16.710670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f1d6-fada-41b0-8dfd-086c36ef38fa None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:16.711243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f1d6-fada-41b0-8dfd-086c36ef38fa None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:16.711243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f1d6-fada-41b0-8dfd-086c36ef38fa None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:16.712532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:16.718919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77758e8-8173-4e63-9b64-5b381e9586ce None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:16.719390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77758e8-8173-4e63-9b64-5b381e9586ce None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:16.720191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77758e8-8173-4e63-9b64-5b381e9586ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:16.796524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a77758e8-8173-4e63-9b64-5b381e9586ce None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:16.798377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a77758e8-8173-4e63-9b64-5b381e9586ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:16.890780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:17.041869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98f1d6-fada-41b0-8dfd-086c36ef38fa None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:17.043448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5e98f1d6-fada-41b0-8dfd-086c36ef38fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:17.091125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:17.190815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98f1d6-fada-41b0-8dfd-086c36ef38fa None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:17.307401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b2492-f709-4986-bf0c-42368a512bc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:17.308001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b2492-f709-4986-bf0c-42368a512bc3 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:17.308277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b2492-f709-4986-bf0c-42368a512bc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:17.308546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b2492-f709-4986-bf0c-42368a512bc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:17.310150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-437b2492-f709-4986-bf0c-42368a512bc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:17.317509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:17.401078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b2492-f709-4986-bf0c-42368a512bc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:17.402715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b2492-f709-4986-bf0c-42368a512bc3 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:17.420328 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 716/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:18.475255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317244a7-4ffa-4e41-b1fe-8eaab5f3f439 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:18.475834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317244a7-4ffa-4e41-b1fe-8eaab5f3f439 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:18.475834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317244a7-4ffa-4e41-b1fe-8eaab5f3f439 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:18.477145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:18.483292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c602-054a-41ad-9e29-d9f17f27a04e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:18.483708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c602-054a-41ad-9e29-d9f17f27a04e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:18.483842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c602-054a-41ad-9e29-d9f17f27a04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:18.559639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4803c602-054a-41ad-9e29-d9f17f27a04e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:18.560856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4803c602-054a-41ad-9e29-d9f17f27a04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:18.627774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:18.655882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:18.806737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317244a7-4ffa-4e41-b1fe-8eaab5f3f439 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:18.808460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-317244a7-4ffa-4e41-b1fe-8eaab5f3f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:18.826070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:18.951321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317244a7-4ffa-4e41-b1fe-8eaab5f3f439 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:19.039720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:19.065899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88829b88-df3d-4f2b-9ced-fb6ff7736e5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:19.066473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88829b88-df3d-4f2b-9ced-fb6ff7736e5a service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:19.066752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88829b88-df3d-4f2b-9ced-fb6ff7736e5a service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:19.067022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88829b88-df3d-4f2b-9ced-fb6ff7736e5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:19.068618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-88829b88-df3d-4f2b-9ced-fb6ff7736e5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:19.156102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88829b88-df3d-4f2b-9ced-fb6ff7736e5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:19.158344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88829b88-df3d-4f2b-9ced-fb6ff7736e5a service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:19.175537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 717/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:20.223173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dfbfd-f7c6-4feb-be81-8bc1d0177592 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:20.223745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dfbfd-f7c6-4feb-be81-8bc1d0177592 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:20.223745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dfbfd-f7c6-4feb-be81-8bc1d0177592 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:20.225037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:20.231220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61d43d-cfd5-42d3-8601-673f76b0fd59 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:20.231638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61d43d-cfd5-42d3-8601-673f76b0fd59 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:20.231762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61d43d-cfd5-42d3-8601-673f76b0fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:20.306606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3a61d43d-cfd5-42d3-8601-673f76b0fd59 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:20.307791 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3a61d43d-cfd5-42d3-8601-673f76b0fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:20.348789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:20.403197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:20.551889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dfbfd-f7c6-4feb-be81-8bc1d0177592 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:20.553346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c78dfbfd-f7c6-4feb-be81-8bc1d0177592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:20.704298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dfbfd-f7c6-4feb-be81-8bc1d0177592 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:20.759462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:20.819267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669d01b-91c0-4784-aae2-7db7c1dbb69b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:20.819792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669d01b-91c0-4784-aae2-7db7c1dbb69b service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:20.820070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669d01b-91c0-4784-aae2-7db7c1dbb69b service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:20.820342 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669d01b-91c0-4784-aae2-7db7c1dbb69b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:20.821972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9669d01b-91c0-4784-aae2-7db7c1dbb69b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:20.909359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669d01b-91c0-4784-aae2-7db7c1dbb69b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:20.910965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669d01b-91c0-4784-aae2-7db7c1dbb69b service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:20.929422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 719/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:22.021263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a4466-41fc-4c84-abce-d25610317b18 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:22.021804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a4466-41fc-4c84-abce-d25610317b18 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:22.021804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a4466-41fc-4c84-abce-d25610317b18 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:22.023160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:22.029463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a40a5-60e1-4634-b933-9c336ace30a4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:22.029872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a40a5-60e1-4634-b933-9c336ace30a4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:22.030086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a40a5-60e1-4634-b933-9c336ace30a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:22.106041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-647a40a5-60e1-4634-b933-9c336ace30a4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:22.107258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-647a40a5-60e1-4634-b933-9c336ace30a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:22.109247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:22.209139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:22.359294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a4466-41fc-4c84-abce-d25610317b18 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:22.360858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-117a4466-41fc-4c84-abce-d25610317b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:22.502081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a4466-41fc-4c84-abce-d25610317b18 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:22.530191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:22.616395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbb9a5-6034-4c98-ad09-d55e109f2c02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:22.616936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbb9a5-6034-4c98-ad09-d55e109f2c02 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:22.617213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbb9a5-6034-4c98-ad09-d55e109f2c02 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:22.617483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbb9a5-6034-4c98-ad09-d55e109f2c02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:22.619095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5ecbb9a5-6034-4c98-ad09-d55e109f2c02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:22.706962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbb9a5-6034-4c98-ad09-d55e109f2c02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:22.708573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbb9a5-6034-4c98-ad09-d55e109f2c02 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:22.726210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 720/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:24.758755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0d3d-c04a-41bf-b767-1dbb8a74b4f3 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:24.759306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0d3d-c04a-41bf-b767-1dbb8a74b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:24.759306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0d3d-c04a-41bf-b767-1dbb8a74b4f3 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:24.760676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:24.765976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e97c9-d7ee-4287-9eaa-1e39da0ec6cd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:24.766426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e97c9-d7ee-4287-9eaa-1e39da0ec6cd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:24.766520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e97c9-d7ee-4287-9eaa-1e39da0ec6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:24.807306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:24.831849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f73e97c9-d7ee-4287-9eaa-1e39da0ec6cd None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:24.833051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f73e97c9-d7ee-4287-9eaa-1e39da0ec6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:24.927538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:25.224431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0d3d-c04a-41bf-b767-1dbb8a74b4f3 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:25.226003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6cfc0d3d-c04a-41bf-b767-1dbb8a74b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:25.326940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0d3d-c04a-41bf-b767-1dbb8a74b4f3 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:25.358859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:25.399900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f61362-5b54-4a6e-a31a-a8b4b7711863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:25.400490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f61362-5b54-4a6e-a31a-a8b4b7711863 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:25.400764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f61362-5b54-4a6e-a31a-a8b4b7711863 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:25.401033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f61362-5b54-4a6e-a31a-a8b4b7711863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:25.402626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-36f61362-5b54-4a6e-a31a-a8b4b7711863 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:25.512534 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f61362-5b54-4a6e-a31a-a8b4b7711863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:25.513875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f61362-5b54-4a6e-a31a-a8b4b7711863 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:25.534124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 722/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:27.820001 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15ed4d-9c13-4f5e-9874-f964e29fa4d1 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:27.820563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15ed4d-9c13-4f5e-9874-f964e29fa4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:27.820563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15ed4d-9c13-4f5e-9874-f964e29fa4d1 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:27.821924 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:27.827230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482050e-d1e9-440f-a5ac-df31782cdfa1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:27.827642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482050e-d1e9-440f-a5ac-df31782cdfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:27.827786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482050e-d1e9-440f-a5ac-df31782cdfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:27.895393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7482050e-d1e9-440f-a5ac-df31782cdfa1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:27.896522 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7482050e-d1e9-440f-a5ac-df31782cdfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:27.909474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:27.992382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:28.280183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15ed4d-9c13-4f5e-9874-f964e29fa4d1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:28.282343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-aa15ed4d-9c13-4f5e-9874-f964e29fa4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:28.383268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15ed4d-9c13-4f5e-9874-f964e29fa4d1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:28.396250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:28.457555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade94ac-2b10-4e7d-baf0-ba4add41d048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:28.458190 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade94ac-2b10-4e7d-baf0-ba4add41d048 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:28.458475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade94ac-2b10-4e7d-baf0-ba4add41d048 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:28.458751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade94ac-2b10-4e7d-baf0-ba4add41d048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:28.460364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5ade94ac-2b10-4e7d-baf0-ba4add41d048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:28.568477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade94ac-2b10-4e7d-baf0-ba4add41d048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:28.569818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade94ac-2b10-4e7d-baf0-ba4add41d048 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:28.587869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 723/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:31.068748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46879a-e2b6-4130-aeb8-943ef4094c24 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:31.069268 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46879a-e2b6-4130-aeb8-943ef4094c24 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:31.069268 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46879a-e2b6-4130-aeb8-943ef4094c24 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:31.070655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:31.075784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef422e-ff6d-4e99-a06c-12010724c9e7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:31.076174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef422e-ff6d-4e99-a06c-12010724c9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:31.076329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef422e-ff6d-4e99-a06c-12010724c9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:31.123975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:31.143639 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9fef422e-ff6d-4e99-a06c-12010724c9e7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:31.144848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9fef422e-ff6d-4e99-a06c-12010724c9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:31.240336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:31.524756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46879a-e2b6-4130-aeb8-943ef4094c24 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:31.526348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4b46879a-e2b6-4130-aeb8-943ef4094c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:31.631606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46879a-e2b6-4130-aeb8-943ef4094c24 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:31.671030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:31.705302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545dcde-716f-4e4e-8de5-8450ed59bb2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:31.706038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545dcde-716f-4e4e-8de5-8450ed59bb2d service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:31.706325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545dcde-716f-4e4e-8de5-8450ed59bb2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:31.706603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545dcde-716f-4e4e-8de5-8450ed59bb2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:31.708237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8545dcde-716f-4e4e-8de5-8450ed59bb2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:31.817209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545dcde-716f-4e4e-8de5-8450ed59bb2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:31.818616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545dcde-716f-4e4e-8de5-8450ed59bb2d service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:31.836440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 725/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:34.117580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cf049-f652-4024-b7dc-5a5df470626f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:34.118174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cf049-f652-4024-b7dc-5a5df470626f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:34.118174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cf049-f652-4024-b7dc-5a5df470626f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:34.119519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:34.124846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cb824-dbc4-487a-9c33-ab15b63829e7 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:34.125229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cb824-dbc4-487a-9c33-ab15b63829e7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:34.125442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cb824-dbc4-487a-9c33-ab15b63829e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:34.192125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-189cb824-dbc4-487a-9c33-ab15b63829e7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:34.193341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-189cb824-dbc4-487a-9c33-ab15b63829e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:34.213926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:34.294521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:34.577505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cf049-f652-4024-b7dc-5a5df470626f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:34.579672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-973cf049-f652-4024-b7dc-5a5df470626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:34.681026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cf049-f652-4024-b7dc-5a5df470626f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:34.701335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:34.754423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02725fc6-c04d-4395-89f4-e05f6e73afa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:34.755017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02725fc6-c04d-4395-89f4-e05f6e73afa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:34.755305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02725fc6-c04d-4395-89f4-e05f6e73afa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:34.755566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02725fc6-c04d-4395-89f4-e05f6e73afa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:34.757144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-02725fc6-c04d-4395-89f4-e05f6e73afa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:34.862374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02725fc6-c04d-4395-89f4-e05f6e73afa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:34.863791 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02725fc6-c04d-4395-89f4-e05f6e73afa6 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:34.881426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 726/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:37.070284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6c5f-ea46-4710-aacd-d1394cff0062 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:37.070793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6c5f-ea46-4710-aacd-d1394cff0062 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:37.070978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6c5f-ea46-4710-aacd-d1394cff0062 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:37.072416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:37.077629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb222b28-b615-42f3-ab79-42a7606a4c61 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:37.078070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb222b28-b615-42f3-ab79-42a7606a4c61 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:37.078262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb222b28-b615-42f3-ab79-42a7606a4c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:37.142883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:37.145554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cb222b28-b615-42f3-ab79-42a7606a4c61 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:37.147294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cb222b28-b615-42f3-ab79-42a7606a4c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:37.243663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:37.524304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6c5f-ea46-4710-aacd-d1394cff0062 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:37.525855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-22bb6c5f-ea46-4710-aacd-d1394cff0062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:37.631040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6c5f-ea46-4710-aacd-d1394cff0062 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:37.679663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:37.705020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570e431-1c3f-471b-bf43-eeb0ac0327a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:37.705619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570e431-1c3f-471b-bf43-eeb0ac0327a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:37.705896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570e431-1c3f-471b-bf43-eeb0ac0327a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:37.706195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570e431-1c3f-471b-bf43-eeb0ac0327a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:37.707780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6570e431-1c3f-471b-bf43-eeb0ac0327a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:37.813653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570e431-1c3f-471b-bf43-eeb0ac0327a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:37.815027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570e431-1c3f-471b-bf43-eeb0ac0327a9 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:37.832845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 728/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:40.017960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc14f4b-5b8c-437d-a413-bd5631a7ff86 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:40.018522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc14f4b-5b8c-437d-a413-bd5631a7ff86 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:40.018522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc14f4b-5b8c-437d-a413-bd5631a7ff86 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:40.019853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:40.025066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54886-f941-450f-bb71-36adab9d5aef None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:40.025475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54886-f941-450f-bb71-36adab9d5aef None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:40.025628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54886-f941-450f-bb71-36adab9d5aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:40.091599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-baf54886-f941-450f-bb71-36adab9d5aef None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:40.092804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-baf54886-f941-450f-bb71-36adab9d5aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:40.122508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:40.189107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:21:40.479677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc14f4b-5b8c-437d-a413-bd5631a7ff86 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:40.481254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ccc14f4b-5b8c-437d-a413-bd5631a7ff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:40.584266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc14f4b-5b8c-437d-a413-bd5631a7ff86 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:40.613378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:40.659281 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d92e8-4fb3-41bf-a427-fb694edccf3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:40.659887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d92e8-4fb3-41bf-a427-fb694edccf3c service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:40.660166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d92e8-4fb3-41bf-a427-fb694edccf3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:40.660458 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d92e8-4fb3-41bf-a427-fb694edccf3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:40.662048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-787d92e8-4fb3-41bf-a427-fb694edccf3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:40.770504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d92e8-4fb3-41bf-a427-fb694edccf3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:40.771869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d92e8-4fb3-41bf-a427-fb694edccf3c service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:40.789992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 729/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:43.025485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c453275-3683-48bf-ae5f-044fd2b492a0 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:43.026035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c453275-3683-48bf-ae5f-044fd2b492a0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:43.026149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c453275-3683-48bf-ae5f-044fd2b492a0 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:43.027492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:43.032865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d84338-8c6f-431e-adfd-fee3bb9349e5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:43.033286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d84338-8c6f-431e-adfd-fee3bb9349e5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:43.033390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d84338-8c6f-431e-adfd-fee3bb9349e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:43.097031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-60d84338-8c6f-431e-adfd-fee3bb9349e5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:43.098244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-60d84338-8c6f-431e-adfd-fee3bb9349e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:43.106299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:43.193789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:43.482772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c453275-3683-48bf-ae5f-044fd2b492a0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:43.484314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9c453275-3683-48bf-ae5f-044fd2b492a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:43.587838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c453275-3683-48bf-ae5f-044fd2b492a0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:43.643916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:43.662227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af49ae-c7b5-4273-9653-31d90ae730f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:43.662835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af49ae-c7b5-4273-9653-31d90ae730f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:43.663116 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af49ae-c7b5-4273-9653-31d90ae730f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:43.663390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af49ae-c7b5-4273-9653-31d90ae730f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:43.664972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c5af49ae-c7b5-4273-9653-31d90ae730f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:43.771230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af49ae-c7b5-4273-9653-31d90ae730f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:43.772572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af49ae-c7b5-4273-9653-31d90ae730f2 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:43.790534 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 731/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:46.000819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b123f-634f-471f-9c25-d42f7ab5add6 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:46.001378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b123f-634f-471f-9c25-d42f7ab5add6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:46.001378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b123f-634f-471f-9c25-d42f7ab5add6 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:46.002685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:46.007825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56edadb-ecaa-43b5-95af-b8dbf6d6bbe0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:46.008281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56edadb-ecaa-43b5-95af-b8dbf6d6bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:46.008383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56edadb-ecaa-43b5-95af-b8dbf6d6bbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:46.073295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f56edadb-ecaa-43b5-95af-b8dbf6d6bbe0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:46.074515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f56edadb-ecaa-43b5-95af-b8dbf6d6bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:46.113560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:46.303519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:46.593655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b123f-634f-471f-9c25-d42f7ab5add6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:46.595289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-587b123f-634f-471f-9c25-d42f7ab5add6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:46.698268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b123f-634f-471f-9c25-d42f7ab5add6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:46.735727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:46.771539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d77105-34db-46f3-9e53-a855fae64bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:46.772135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d77105-34db-46f3-9e53-a855fae64bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:46.772412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d77105-34db-46f3-9e53-a855fae64bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:46.772683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d77105-34db-46f3-9e53-a855fae64bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:46.774298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-34d77105-34db-46f3-9e53-a855fae64bc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:46.888594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d77105-34db-46f3-9e53-a855fae64bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:46.890223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d77105-34db-46f3-9e53-a855fae64bc4 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:47.025298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 732/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:49.199439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c0f64-58f7-4328-874f-61edec739495 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:49.199970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c0f64-58f7-4328-874f-61edec739495 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:49.199970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c0f64-58f7-4328-874f-61edec739495 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:49.201290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:49.207090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0177d-a147-4f72-a0e4-341582439d3c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:49.207510 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0177d-a147-4f72-a0e4-341582439d3c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:49.207614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0177d-a147-4f72-a0e4-341582439d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:49.272811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-99e0177d-a147-4f72-a0e4-341582439d3c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:49.273922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-99e0177d-a147-4f72-a0e4-341582439d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:49.291920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:49.368137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:49.647516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c0f64-58f7-4328-874f-61edec739495 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:49.649104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9b4c0f64-58f7-4328-874f-61edec739495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:49.750742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c0f64-58f7-4328-874f-61edec739495 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:49.818836 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:49.825726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3052e1-1d6a-4768-ad19-f15bb19c6d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:49.826355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3052e1-1d6a-4768-ad19-f15bb19c6d7f service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:49.826631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3052e1-1d6a-4768-ad19-f15bb19c6d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:49.826899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3052e1-1d6a-4768-ad19-f15bb19c6d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:49.829054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2d3052e1-1d6a-4768-ad19-f15bb19c6d7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:49.934591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3052e1-1d6a-4768-ad19-f15bb19c6d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:49.935977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3052e1-1d6a-4768-ad19-f15bb19c6d7f service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:49.953891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 734/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:21:52.139943 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a6dd-feb0-4641-ad50-d125dca49971 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:52.141088 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a6dd-feb0-4641-ad50-d125dca49971 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:52.141210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a6dd-feb0-4641-ad50-d125dca49971 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:52.142560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:21:52.147913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b330ce-82cf-41ad-9774-fa5a177bbb5c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:52.148349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b330ce-82cf-41ad-9774-fa5a177bbb5c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:52.148454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b330ce-82cf-41ad-9774-fa5a177bbb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:52.213785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-50b330ce-82cf-41ad-9774-fa5a177bbb5c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:21:52.215396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-50b330ce-82cf-41ad-9774-fa5a177bbb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:52.264546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:52.314441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:21:52.601543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66a6dd-feb0-4641-ad50-d125dca49971 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:21:52.603147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0c66a6dd-feb0-4641-ad50-d125dca49971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:52.704454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66a6dd-feb0-4641-ad50-d125dca49971 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:21:52.752053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:21:52.778523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8865707-3473-4947-96e4-33d10ef944a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:21:52.779114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8865707-3473-4947-96e4-33d10ef944a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:21:52.779387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8865707-3473-4947-96e4-33d10ef944a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:21:52.779652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8865707-3473-4947-96e4-33d10ef944a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:21:52.781238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f8865707-3473-4947-96e4-33d10ef944a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:21:52.886575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8865707-3473-4947-96e4-33d10ef944a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:21:52.888184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8865707-3473-4947-96e4-33d10ef944a1 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:21:52.906266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 735/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:22:02.312698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=22 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:02.904092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=66 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:12.757046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff79e7-4cb3-4ebc-a0ee-b5e4fab167bb None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:12.757607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff79e7-4cb3-4ebc-a0ee-b5e4fab167bb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:12.757607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff79e7-4cb3-4ebc-a0ee-b5e4fab167bb None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:12.758951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 736/1471] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:22:12.764097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38037636-c7bc-438c-bb1d-0df1b36a99bc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:12.764518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38037636-c7bc-438c-bb1d-0df1b36a99bc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:12.765153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38037636-c7bc-438c-bb1d-0df1b36a99bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:12.830533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-38037636-c7bc-438c-bb1d-0df1b36a99bc None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:22:12.831771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-38037636-c7bc-438c-bb1d-0df1b36a99bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:12.925495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 736/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:22:13.225630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3f044-d933-4029-9395-e3064b84d38d None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:13.225913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3f044-d933-4029-9395-e3064b84d38d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:13.226221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3f044-d933-4029-9395-e3064b84d38d None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:13.227580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 737/1473] 10.222.0.18 () {54 vars in 962 bytes} [Wed Sep 21 05:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:22:13.232546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c24f99-f378-4024-bc7c-d67dae689e7b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:13.232830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c24f99-f378-4024-bc7c-d67dae689e7b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:13.233105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c24f99-f378-4024-bc7c-d67dae689e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:13.307956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b9c24f99-f378-4024-bc7c-d67dae689e7b None None] MFA Rules not processed for user `5305f8a5d2b540de91d3caedad454bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:22:13.309164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b9c24f99-f378-4024-bc7c-d67dae689e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:13.329822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:13.384648 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 737/1474] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Sep 21 05:22:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:22:13.392411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3f044-d933-4029-9395-e3064b84d38d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:22:13.393992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5da3f044-d933-4029-9395-e3064b84d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:13.496974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3f044-d933-4029-9395-e3064b84d38d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:22:13.569309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05548e39-bcdb-4f95-9a7e-ef1bb3037358 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5305f8a5d2b540de91d3caedad454bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:22:13.569907 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548e39-bcdb-4f95-9a7e-ef1bb3037358 service glance] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:13.570219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548e39-bcdb-4f95-9a7e-ef1bb3037358 service glance] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:13.570490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548e39-bcdb-4f95-9a7e-ef1bb3037358 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:13.572071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-05548e39-bcdb-4f95-9a7e-ef1bb3037358 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:13.628617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:13.682969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05548e39-bcdb-4f95-9a7e-ef1bb3037358 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:22:13.684392 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05548e39-bcdb-4f95-9a7e-ef1bb3037358 service glance] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:22:13.702150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 738/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:22:15.956348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c6690-556b-48ee-8c5d-16d698351746 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:15.956881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c6690-556b-48ee-8c5d-16d698351746 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:15.956881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c6690-556b-48ee-8c5d-16d698351746 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:15.958186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 738/1476] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:22:15.963404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6285beb-531f-40f8-b096-44dec9a0ad56 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:15.963828 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6285beb-531f-40f8-b096-44dec9a0ad56 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:15.963963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6285beb-531f-40f8-b096-44dec9a0ad56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:16.029337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b6285beb-531f-40f8-b096-44dec9a0ad56 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:22:16.031062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b6285beb-531f-40f8-b096-44dec9a0ad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:16.126897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 739/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:22:16.403996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c6690-556b-48ee-8c5d-16d698351746 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:22:16.406229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-299c6690-556b-48ee-8c5d-16d698351746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:16.531059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c6690-556b-48ee-8c5d-16d698351746 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:22:16.578829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:16.622896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a9bd2-3b9a-48ef-9969-c0f8ffb39232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:22:16.623493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a9bd2-3b9a-48ef-9969-c0f8ffb39232 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:16.623769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a9bd2-3b9a-48ef-9969-c0f8ffb39232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:16.624068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a9bd2-3b9a-48ef-9969-c0f8ffb39232 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:16.624701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a9bd2-3b9a-48ef-9969-c0f8ffb39232 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:22:16.626459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-a96a9bd2-3b9a-48ef-9969-c0f8ffb39232 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:22:16.626459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Sep 21 05:22:16.626459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: Sep 21 05:22:16.627584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a9bd2-3b9a-48ef-9969-c0f8ffb39232 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:22:16.646099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 739/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Sep 21 05:22:16 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:22:18.854042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763f61-2dc6-45aa-a880-d2af59d6dc59 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:18.854592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763f61-2dc6-45aa-a880-d2af59d6dc59 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:18.854592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763f61-2dc6-45aa-a880-d2af59d6dc59 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:18.855843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 740/1479] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:22:18.861658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7934-345a-4b5a-bd49-a907f49dff4c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:18.862345 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7934-345a-4b5a-bd49-a907f49dff4c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:18.862497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7934-345a-4b5a-bd49-a907f49dff4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:18.928720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-546f7934-345a-4b5a-bd49-a907f49dff4c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:22:18.929996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-546f7934-345a-4b5a-bd49-a907f49dff4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:19.023769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 740/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:22:19.319713 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af763f61-2dc6-45aa-a880-d2af59d6dc59 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:22:19.321413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-af763f61-2dc6-45aa-a880-d2af59d6dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:19.331728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:19.464172 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af763f61-2dc6-45aa-a880-d2af59d6dc59 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:22:19.544084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:19.575890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e2e27-fa4e-422b-8409-820e39f8e465 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:22:19.576420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e2e27-fa4e-422b-8409-820e39f8e465 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:19.576694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e2e27-fa4e-422b-8409-820e39f8e465 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:19.577539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e2e27-fa4e-422b-8409-820e39f8e465 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:19.579142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2a6e2e27-fa4e-422b-8409-820e39f8e465 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:19.667827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e2e27-fa4e-422b-8409-820e39f8e465 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:22:19.669402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e2e27-fa4e-422b-8409-820e39f8e465 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:22:19.687296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 741/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:22:22.031836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6773f6-d462-4c3c-98da-e8846f2c3510 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:22.032380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6773f6-d462-4c3c-98da-e8846f2c3510 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:22.032380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6773f6-d462-4c3c-98da-e8846f2c3510 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:22.033626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 741/1482] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:22:22.038827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e564c-bdec-4758-a162-ebe883d9091a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:22.039266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e564c-bdec-4758-a162-ebe883d9091a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:22.039370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e564c-bdec-4758-a162-ebe883d9091a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:22.108692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f45e564c-bdec-4758-a162-ebe883d9091a None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:22:22.109901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f45e564c-bdec-4758-a162-ebe883d9091a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:22.156712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:22.206453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 742/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:22:22.496927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6773f6-d462-4c3c-98da-e8846f2c3510 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:22:22.498493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5e6773f6-d462-4c3c-98da-e8846f2c3510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:22.522512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:22.603734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6773f6-d462-4c3c-98da-e8846f2c3510 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:22:22.677108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870cdbe-5486-48bb-a917-a28719da153e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:22:22.677701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870cdbe-5486-48bb-a917-a28719da153e service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:22.678002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870cdbe-5486-48bb-a917-a28719da153e service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:22.678344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870cdbe-5486-48bb-a917-a28719da153e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:22.679931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2870cdbe-5486-48bb-a917-a28719da153e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:22.755761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:22.795227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870cdbe-5486-48bb-a917-a28719da153e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:22:22.797321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870cdbe-5486-48bb-a917-a28719da153e service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:22:22.814825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 742/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:22:25.007354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e166b-ce62-4ade-9e1e-a08e21e50312 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:25.007820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e166b-ce62-4ade-9e1e-a08e21e50312 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:25.007919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e166b-ce62-4ade-9e1e-a08e21e50312 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:25.009232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 743/1485] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:22:25.015315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b16a97-4a56-4bf1-b923-9f9eed56c7db None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:25.015725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b16a97-4a56-4bf1-b923-9f9eed56c7db None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:25.015894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b16a97-4a56-4bf1-b923-9f9eed56c7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:25.081018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-00b16a97-4a56-4bf1-b923-9f9eed56c7db None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:22:25.082254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-00b16a97-4a56-4bf1-b923-9f9eed56c7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:25.176552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 743/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:22:25.466160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e166b-ce62-4ade-9e1e-a08e21e50312 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:22:25.467792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-568e166b-ce62-4ade-9e1e-a08e21e50312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:25.571953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e166b-ce62-4ade-9e1e-a08e21e50312 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:22:25.644659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:25.646213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6835f27-6ec0-4976-88df-d8cf377c5b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:22:25.646879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6835f27-6ec0-4976-88df-d8cf377c5b91 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:25.647181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6835f27-6ec0-4976-88df-d8cf377c5b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:25.647463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6835f27-6ec0-4976-88df-d8cf377c5b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:25.649569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d6835f27-6ec0-4976-88df-d8cf377c5b91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:25.756784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6835f27-6ec0-4976-88df-d8cf377c5b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:22:25.758154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6835f27-6ec0-4976-88df-d8cf377c5b91 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:22:25.775532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 744/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:22:28.081384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1cd61-120c-4863-98d4-0e5ec8787bb0 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:28.081902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1cd61-120c-4863-98d4-0e5ec8787bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:28.082047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1cd61-120c-4863-98d4-0e5ec8787bb0 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:28.083314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 744/1488] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:22:28.088593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b937d97-86c3-4812-9fb9-f72cf2b92e24 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:28.089034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b937d97-86c3-4812-9fb9-f72cf2b92e24 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:28.089129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b937d97-86c3-4812-9fb9-f72cf2b92e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:28.154528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6b937d97-86c3-4812-9fb9-f72cf2b92e24 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:22:28.156246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6b937d97-86c3-4812-9fb9-f72cf2b92e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:28.206766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:28.251081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 745/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:22:28.550899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1cd61-120c-4863-98d4-0e5ec8787bb0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:22:28.552446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-83d1cd61-120c-4863-98d4-0e5ec8787bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:28.587784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:28.656327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1cd61-120c-4863-98d4-0e5ec8787bb0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:22:28.728226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46011d96-051f-4b3e-9133-139dcd191c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:22:28.728804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46011d96-051f-4b3e-9133-139dcd191c25 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:28.729080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46011d96-051f-4b3e-9133-139dcd191c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:28.729348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46011d96-051f-4b3e-9133-139dcd191c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:28.730956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-46011d96-051f-4b3e-9133-139dcd191c25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:28.810830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:28.838867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46011d96-051f-4b3e-9133-139dcd191c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:22:28.840324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46011d96-051f-4b3e-9133-139dcd191c25 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:22:28.858849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 745/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:22:31.164969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd2f2e-2746-4ef9-ba41-35b4059de76d None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:31.165561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd2f2e-2746-4ef9-ba41-35b4059de76d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:31.165561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd2f2e-2746-4ef9-ba41-35b4059de76d None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:31.166917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 746/1491] 10.222.0.18 () {54 vars in 927 bytes} [Wed Sep 21 05:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:22:31.172545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddb69-0bd9-42f9-89d5-e22b092bbaf4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:31.172992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddb69-0bd9-42f9-89d5-e22b092bbaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:31.173099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddb69-0bd9-42f9-89d5-e22b092bbaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:31.238207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e1eddb69-0bd9-42f9-89d5-e22b092bbaf4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:22:31.239408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e1eddb69-0bd9-42f9-89d5-e22b092bbaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:31.337448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 746/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Sep 21 05:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:22:31.635787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd2f2e-2746-4ef9-ba41-35b4059de76d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:22:31.637323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0fdd2f2e-2746-4ef9-ba41-35b4059de76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:31.740017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd2f2e-2746-4ef9-ba41-35b4059de76d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:22:31.810966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa28f99-0be7-4d56-a7b6-cd1cf63882d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:22:31.811580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa28f99-0be7-4d56-a7b6-cd1cf63882d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:22:31.811864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa28f99-0be7-4d56-a7b6-cd1cf63882d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:22:31.812171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa28f99-0be7-4d56-a7b6-cd1cf63882d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:22:31.813831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4fa28f99-0be7-4d56-a7b6-cd1cf63882d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:22:31.823136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:31.929250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa28f99-0be7-4d56-a7b6-cd1cf63882d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:22:31.930621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa28f99-0be7-4d56-a7b6-cd1cf63882d4 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:22:31.948256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 747/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:22:41.335926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=82 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:22:41.946080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=50 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:51.431430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045f4ec-2624-4c79-98d8-56a03c085442 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:51.432012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045f4ec-2624-4c79-98d8-56a03c085442 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:51.432012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045f4ec-2624-4c79-98d8-56a03c085442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:51.506507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e045f4ec-2624-4c79-98d8-56a03c085442 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:23:51.507795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e045f4ec-2624-4c79-98d8-56a03c085442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:51.601513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 747/1494] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:23:51.609951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a043b2-dc60-4ff4-8af8-de64a77dd56d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:51.610388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a043b2-dc60-4ff4-8af8-de64a77dd56d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:51.610505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a043b2-dc60-4ff4-8af8-de64a77dd56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:51.685102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-70a043b2-dc60-4ff4-8af8-de64a77dd56d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:23:51.686247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-70a043b2-dc60-4ff4-8af8-de64a77dd56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:51.781103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 748/1495] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:23:51.785723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045f4ec-2624-4c79-98d8-56a03c085442 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:51.786962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e045f4ec-2624-4c79-98d8-56a03c085442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:51.798851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045f4ec-2624-4c79-98d8-56a03c085442 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:51.859927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:51.894479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:51.895073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:51.895350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:51.895617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:51.896246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:23:51.896549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:51.897552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:23:51.897552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Sep 21 05:23:51.897552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: Sep 21 05:23:51.898462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:51.909767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 748/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:23:51.918657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a043b2-dc60-4ff4-8af8-de64a77dd56d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:51.927606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a043b2-dc60-4ff4-8af8-de64a77dd56d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:51.988882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:52.021450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:52.022118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:52.022396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:52.022664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:52.023290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:52.025003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:23:52.025003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 21 05:23:52.025003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: Sep 21 05:23:52.026196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:52.080711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 749/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:23:51 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:23:52.085090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:52.086646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:52.120424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdebcb-7871-4011-bcfc-fe02dee2f8d6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:52.211794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:52.212396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:52.212670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:52.212937 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:52.213573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:52.215288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:23:52.215288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Sep 21 05:23:52.215288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: Sep 21 05:23:52.216562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:52.260065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 749/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:23:52 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:23:52.264785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:52.266334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:52.327285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42336436-1066-4f6b-8495-4e05e67dce0f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:52.410767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:52.421746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:52.422398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:52.422675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:52.422944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:52.423567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:52.424588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:23:52.424588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 21 05:23:52.424588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: Sep 21 05:23:52.425460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:52.437431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 750/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:23:52.441586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:52.443191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:52.454939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18969e-12fa-4981-a5b1-7c59abfe7ec5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:52.479667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:52.548947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e26cd-3645-413e-9dd1-8dbbb818680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:52.549467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e26cd-3645-413e-9dd1-8dbbb818680d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:52.549746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e26cd-3645-413e-9dd1-8dbbb818680d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:52.550042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e26cd-3645-413e-9dd1-8dbbb818680d admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:52.550669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e26cd-3645-413e-9dd1-8dbbb818680d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:52.551680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-832e26cd-3645-413e-9dd1-8dbbb818680d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:23:52.551680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Sep 21 05:23:52.551680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: Sep 21 05:23:52.552558 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e26cd-3645-413e-9dd1-8dbbb818680d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:52.568571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 750/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:23:52.569880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:52.571425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:52.605378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917351b0-f482-46eb-9c5d-6007f7fb7942 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:52.697354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527509ad-a97d-4d10-8e49-147de394947e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:52.698028 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527509ad-a97d-4d10-8e49-147de394947e admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:52.698311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527509ad-a97d-4d10-8e49-147de394947e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:52.698583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527509ad-a97d-4d10-8e49-147de394947e admin admin] PATH_INFO: `/v3/projects/2f01589e1b7746eb858466a56b3baf68/users/691c09242b854008900b9bab4b7ad999/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:52.720857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527509ad-a97d-4d10-8e49-147de394947e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f01589e1b7746eb858466a56b3baf68, user_id=691c09242b854008900b9bab4b7ad999, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:52.721880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-527509ad-a97d-4d10-8e49-147de394947e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:23:52.721880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 21 05:23:52.721880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: Sep 21 05:23:52.723830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527509ad-a97d-4d10-8e49-147de394947e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:52.747832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 751/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:23:52 2022] PUT /identity/v3/projects/2f01589e1b7746eb858466a56b3baf68/users/691c09242b854008900b9bab4b7ad999/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:23:52.759808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde55e0b-4ad1-448c-874f-e5d2a2a9280f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:52.760093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde55e0b-4ad1-448c-874f-e5d2a2a9280f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:52.760369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde55e0b-4ad1-448c-874f-e5d2a2a9280f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:52.817275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-dde55e0b-4ad1-448c-874f-e5d2a2a9280f None None] MFA Rules not processed for user `691c09242b854008900b9bab4b7ad999`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:23:52.818424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dde55e0b-4ad1-448c-874f-e5d2a2a9280f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:52.894112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 751/1502] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:23:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:23:52.907184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac80363-dbef-4141-b6ab-725d4585260b None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:52.907476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac80363-dbef-4141-b6ab-725d4585260b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:52.907751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac80363-dbef-4141-b6ab-725d4585260b None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:52.909076 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 752/1503] 10.222.0.18 () {54 vars in 962 bytes} [Wed Sep 21 05:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:23:52.913649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d8820-d5a3-41c2-80b8-171c06aa5241 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:52.913951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d8820-d5a3-41c2-80b8-171c06aa5241 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:52.914239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d8820-d5a3-41c2-80b8-171c06aa5241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:52.945426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:52.989709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-093d8820-d5a3-41c2-80b8-171c06aa5241 None None] MFA Rules not processed for user `63a04998aafd40cc941283700898c92f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:23:52.990922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-093d8820-d5a3-41c2-80b8-171c06aa5241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:53.063784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 752/1504] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Sep 21 05:23:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:23:53.071598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac80363-dbef-4141-b6ab-725d4585260b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:53.073050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1ac80363-dbef-4141-b6ab-725d4585260b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:53.124224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:53.177661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac80363-dbef-4141-b6ab-725d4585260b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:53.249897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:53.250535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:53.250816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:53.251091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:53.252682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:53.349561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:53.351014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:53.354466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:53.370167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 753/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:23:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:23:53.414539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d2cee-fa5b-49a4-83be-a4d86735dee2 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:53.414828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d2cee-fa5b-49a4-83be-a4d86735dee2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:53.415103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d2cee-fa5b-49a4-83be-a4d86735dee2 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:53.416349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 753/1506] 10.222.0.18 () {52 vars in 830 bytes} [Wed Sep 21 05:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:23:53.420632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:53.421839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:53.518632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce60b6-ce32-4760-ab2b-de9364617932 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:53.614528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:53.615100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:53.615377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:53.615648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] PATH_INFO: `/v3/users/691c09242b854008900b9bab4b7ad999` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:53.628848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] RBAC: Authorizing `identity:delete_user(user_id=691c09242b854008900b9bab4b7ad999)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:53.629856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:23:53.629856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 21 05:23:53.629856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: Sep 21 05:23:53.631080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:53.641348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:53.696345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691c09242b854008900b9bab4b7ad999'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:53.706304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691c09242b854008900b9bab4b7ad999'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:53.706896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691c09242b854008900b9bab4b7ad999'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:53.707216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691c09242b854008900b9bab4b7ad999'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:53.721471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691c09242b854008900b9bab4b7ad999'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:53.749440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 754/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:23:53 2022] DELETE /identity/v3/users/691c09242b854008900b9bab4b7ad999 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:23:53.753894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc3676-c435-407f-a8ee-e96d3ed6e47c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:53.754381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc3676-c435-407f-a8ee-e96d3ed6e47c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:53.754503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc3676-c435-407f-a8ee-e96d3ed6e47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:53.810469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-bddc3676-c435-407f-a8ee-e96d3ed6e47c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:23:53.811712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bddc3676-c435-407f-a8ee-e96d3ed6e47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:53.887042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:53.899496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 754/1508] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:23:53.909506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:53.910751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:54.006472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bfcf5-1fb7-4a85-b4e2-705f875871da admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:54.061088 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:23:54.081041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd6e5a-bcb6-4f83-a44b-ab1f5d8dd1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:54.081637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd6e5a-bcb6-4f83-a44b-ab1f5d8dd1ad service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:54.081914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd6e5a-bcb6-4f83-a44b-ab1f5d8dd1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:54.082215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd6e5a-bcb6-4f83-a44b-ab1f5d8dd1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:54.083779 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c6bd6e5a-bcb6-4f83-a44b-ab1f5d8dd1ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:54.193904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd6e5a-bcb6-4f83-a44b-ab1f5d8dd1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:54.195266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd6e5a-bcb6-4f83-a44b-ab1f5d8dd1ad service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:54.213268 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 755/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:23:54.561648 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc3676-c435-407f-a8ee-e96d3ed6e47c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:23:54.562960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bddc3676-c435-407f-a8ee-e96d3ed6e47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:23:54.574343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc3676-c435-407f-a8ee-e96d3ed6e47c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:23:54.670609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:23:54.671230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:23:54.671503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:23:54.671770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] PATH_INFO: `/v3/projects/2f01589e1b7746eb858466a56b3baf68` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:23:54.681471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f01589e1b7746eb858466a56b3baf68)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:23:54.682528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:23:54.682528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Sep 21 05:23:54.682528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: Sep 21 05:23:54.683708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:23:54.740473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f01589e1b7746eb858466a56b3baf68', 'request_id': 'req-e70edeab-ba40-4137-8cbb-8e23c8ad1180'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:54.746421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f01589e1b7746eb858466a56b3baf68', 'request_id': 'req-e70edeab-ba40-4137-8cbb-8e23c8ad1180'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:54.756433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f01589e1b7746eb858466a56b3baf68', 'request_id': 'req-e70edeab-ba40-4137-8cbb-8e23c8ad1180'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:54.789610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] The token cache is being invalidate because project 2f01589e1b7746eb858466a56b3baf68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:23:54.789952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:54.790530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e70edeab-ba40-4137-8cbb-8e23c8ad1180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:23:54.791867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 755/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:23:54 2022] DELETE /identity/v3/projects/2f01589e1b7746eb858466a56b3baf68 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:24:04.211049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=62 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:24:04.787448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:25:17.902623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52040f-98fb-400e-b0b2-809b894c6f37 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:17.903168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52040f-98fb-400e-b0b2-809b894c6f37 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:17.903168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52040f-98fb-400e-b0b2-809b894c6f37 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:17.904441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:25:17.909481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d21eee-32eb-4c3f-9f43-0a1798c86245 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:17.909922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d21eee-32eb-4c3f-9f43-0a1798c86245 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:17.910059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d21eee-32eb-4c3f-9f43-0a1798c86245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:17.991334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f1d21eee-32eb-4c3f-9f43-0a1798c86245 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:17.992625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f1d21eee-32eb-4c3f-9f43-0a1798c86245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:18.096323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:25:18.250743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52040f-98fb-400e-b0b2-809b894c6f37 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:25:18.252815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2c52040f-98fb-400e-b0b2-809b894c6f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:18.369000 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52040f-98fb-400e-b0b2-809b894c6f37 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:25:18.374503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc579bf3-43f3-41c1-9368-b521b84e0010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:25:18.375104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579bf3-43f3-41c1-9368-b521b84e0010 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:18.375387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579bf3-43f3-41c1-9368-b521b84e0010 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:18.375663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579bf3-43f3-41c1-9368-b521b84e0010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:18.383557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc579bf3-43f3-41c1-9368-b521b84e0010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:25:18.385121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc579bf3-43f3-41c1-9368-b521b84e0010 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:25:18.394315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 757/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:25:19.476228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33656e09-6c0e-49fe-958a-1497a1d529ca None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:19.476751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33656e09-6c0e-49fe-958a-1497a1d529ca None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:19.476751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33656e09-6c0e-49fe-958a-1497a1d529ca None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:19.478042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:25:19.484564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e25ff-92d1-4289-9762-ccfd7d9f249b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:19.484999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e25ff-92d1-4289-9762-ccfd7d9f249b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:19.485112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e25ff-92d1-4289-9762-ccfd7d9f249b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:19.505515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b32e25ff-92d1-4289-9762-ccfd7d9f249b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:19.506751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b32e25ff-92d1-4289-9762-ccfd7d9f249b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:19.514652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:25:19.673681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33656e09-6c0e-49fe-958a-1497a1d529ca None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:25:19.689045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33656e09-6c0e-49fe-958a-1497a1d529ca None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:25:19.694755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852baa82-87da-4080-b000-ed2d2bdb992b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:25:19.695419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852baa82-87da-4080-b000-ed2d2bdb992b service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:19.695696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852baa82-87da-4080-b000-ed2d2bdb992b service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:19.695967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852baa82-87da-4080-b000-ed2d2bdb992b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:19.704601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852baa82-87da-4080-b000-ed2d2bdb992b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:25:19.706208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852baa82-87da-4080-b000-ed2d2bdb992b service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:25:19.715579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 758/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:25:20.203037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9cd26-260b-4659-a406-9f056b2f32e5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:20.203323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9cd26-260b-4659-a406-9f056b2f32e5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:20.203598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9cd26-260b-4659-a406-9f056b2f32e5 None None] PATH_INFO: `/v3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:20.204872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 759/1517] 10.222.0.52 () {54 vars in 925 bytes} [Wed Sep 21 05:25:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:25:20.211059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95868266-ec02-4538-a637-0ad31faf2eb7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:20.211361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95868266-ec02-4538-a637-0ad31faf2eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:20.211633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95868266-ec02-4538-a637-0ad31faf2eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:20.270331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-95868266-ec02-4538-a637-0ad31faf2eb7 None None] MFA Rules not processed for user `4b3d8a1c8b6640ae9e90ece0bbfbf410`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:20.271527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-95868266-ec02-4538-a637-0ad31faf2eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:20.290610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:25:20.344151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 759/1518] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 21 05:25:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:25:23.900019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe917a3-1e15-4bcb-ab34-b88d09e3ea75 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:23.900543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe917a3-1e15-4bcb-ab34-b88d09e3ea75 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:23.900832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe917a3-1e15-4bcb-ab34-b88d09e3ea75 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:23.902201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 760/1519] 10.222.0.18 () {54 vars in 961 bytes} [Wed Sep 21 05:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:25:23.908163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987aa13c-ac71-49a7-a6dc-8d17f50c776e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:23.908695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987aa13c-ac71-49a7-a6dc-8d17f50c776e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:23.908813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987aa13c-ac71-49a7-a6dc-8d17f50c776e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:23.930857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-987aa13c-ac71-49a7-a6dc-8d17f50c776e None None] MFA Rules not processed for user `4b3d8a1c8b6640ae9e90ece0bbfbf410`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:23.932083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-987aa13c-ac71-49a7-a6dc-8d17f50c776e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:23.940519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 760/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Sep 21 05:25:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:25:23.949194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe917a3-1e15-4bcb-ab34-b88d09e3ea75 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:25:23.963453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe917a3-1e15-4bcb-ab34-b88d09e3ea75 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:25:23.969445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4b62c-578c-4444-86a3-e6b647e571a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:25:23.970078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b62c-578c-4444-86a3-e6b647e571a9 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:23.970361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b62c-578c-4444-86a3-e6b647e571a9 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:23.970633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b62c-578c-4444-86a3-e6b647e571a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:23.979843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4b62c-578c-4444-86a3-e6b647e571a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:25:23.981463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4b62c-578c-4444-86a3-e6b647e571a9 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:25:23.991447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 761/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:25:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:25:31.191367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339ebbc-6586-4765-9f37-abacf68e25f6 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:31.191977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339ebbc-6586-4765-9f37-abacf68e25f6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:31.191977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339ebbc-6586-4765-9f37-abacf68e25f6 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:31.193237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:25:31.199274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a2fef-1458-4b5e-917c-1f78ae5aeff3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:31.199683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a2fef-1458-4b5e-917c-1f78ae5aeff3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:31.199824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a2fef-1458-4b5e-917c-1f78ae5aeff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:31.219096 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b53a2fef-1458-4b5e-917c-1f78ae5aeff3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:31.220278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b53a2fef-1458-4b5e-917c-1f78ae5aeff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:31.227892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:25:31.397423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339ebbc-6586-4765-9f37-abacf68e25f6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:25:31.411985 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339ebbc-6586-4765-9f37-abacf68e25f6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:25:31.417306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cbca8-eb65-4ad0-94a7-370c01489670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:25:31.417921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cbca8-eb65-4ad0-94a7-370c01489670 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:31.418223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cbca8-eb65-4ad0-94a7-370c01489670 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:31.418489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cbca8-eb65-4ad0-94a7-370c01489670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:31.427158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cbca8-eb65-4ad0-94a7-370c01489670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:25:31.428717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cbca8-eb65-4ad0-94a7-370c01489670 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:25:31.438418 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 762/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:25:32.501596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee23fd-e445-450e-922f-faf01913b50f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:32.502216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee23fd-e445-450e-922f-faf01913b50f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:32.502216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee23fd-e445-450e-922f-faf01913b50f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:32.503483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 05:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:25:32.509440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caae9a-55b2-4b3b-b837-6e8797c7c189 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:32.509885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caae9a-55b2-4b3b-b837-6e8797c7c189 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:32.510021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caae9a-55b2-4b3b-b837-6e8797c7c189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:32.530083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-95caae9a-55b2-4b3b-b837-6e8797c7c189 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:32.531303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-95caae9a-55b2-4b3b-b837-6e8797c7c189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:32.539115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 05:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:25:32.690365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee23fd-e445-450e-922f-faf01913b50f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:25:32.704994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee23fd-e445-450e-922f-faf01913b50f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:25:32.710630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396492c-681c-4ee2-820c-75decb09ce40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:25:32.711225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396492c-681c-4ee2-820c-75decb09ce40 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:32.711505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396492c-681c-4ee2-820c-75decb09ce40 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:32.711774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396492c-681c-4ee2-820c-75decb09ce40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:32.719846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396492c-681c-4ee2-820c-75decb09ce40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:25:32.721416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396492c-681c-4ee2-820c-75decb09ce40 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:25:32.730589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 764/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:25:36.536664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301ced6-bceb-4107-99b2-3ca1241ad12f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:36.537287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301ced6-bceb-4107-99b2-3ca1241ad12f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:36.537287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301ced6-bceb-4107-99b2-3ca1241ad12f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:36.538601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:25:36.543754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee8001-6d30-43d2-8088-9e54814b4846 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:36.544155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee8001-6d30-43d2-8088-9e54814b4846 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:36.544317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee8001-6d30-43d2-8088-9e54814b4846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:36.563890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b7ee8001-6d30-43d2-8088-9e54814b4846 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:36.565090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b7ee8001-6d30-43d2-8088-9e54814b4846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:36.572981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:25:36.872958 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301ced6-bceb-4107-99b2-3ca1241ad12f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:25:36.874650 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a301ced6-bceb-4107-99b2-3ca1241ad12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:36.957099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301ced6-bceb-4107-99b2-3ca1241ad12f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:25:36.960729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43dcd5-d0a2-4966-ae85-4b8f8adef4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:25:36.961328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43dcd5-d0a2-4966-ae85-4b8f8adef4ef service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:36.961603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43dcd5-d0a2-4966-ae85-4b8f8adef4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:36.961872 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43dcd5-d0a2-4966-ae85-4b8f8adef4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:36.972258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43dcd5-d0a2-4966-ae85-4b8f8adef4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:25:36.974363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43dcd5-d0a2-4966-ae85-4b8f8adef4ef service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:25:36.985235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 765/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:25:39.125253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65545fba-831e-4b60-a09f-35e2283c9bb4 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:39.125718 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65545fba-831e-4b60-a09f-35e2283c9bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:39.125821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65545fba-831e-4b60-a09f-35e2283c9bb4 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:39.127159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:25:39.132472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770ae20-4584-4f7c-b8e3-b76ad669df38 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:39.132896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770ae20-4584-4f7c-b8e3-b76ad669df38 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:39.133031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770ae20-4584-4f7c-b8e3-b76ad669df38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:39.153070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8770ae20-4584-4f7c-b8e3-b76ad669df38 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:39.155009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8770ae20-4584-4f7c-b8e3-b76ad669df38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:39.162681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:25:39.448861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65545fba-831e-4b60-a09f-35e2283c9bb4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:25:39.451237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-65545fba-831e-4b60-a09f-35e2283c9bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:39.465587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65545fba-831e-4b60-a09f-35e2283c9bb4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:25:39.469115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac623d28-a3b1-4c37-b8f0-cd95855c7820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:25:39.469734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623d28-a3b1-4c37-b8f0-cd95855c7820 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:39.470045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623d28-a3b1-4c37-b8f0-cd95855c7820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:39.470316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623d28-a3b1-4c37-b8f0-cd95855c7820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:39.479184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac623d28-a3b1-4c37-b8f0-cd95855c7820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:25:39.480601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac623d28-a3b1-4c37-b8f0-cd95855c7820 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:25:39.490812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 767/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:25:43.069384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7911f-97d4-4bdf-a29a-1c2a1b7f7967 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:43.069917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7911f-97d4-4bdf-a29a-1c2a1b7f7967 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:43.069917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7911f-97d4-4bdf-a29a-1c2a1b7f7967 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:43.071197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:25:43.076349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a448a9-c8ef-4fa4-8b61-a27ecaffc8a1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:43.076795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a448a9-c8ef-4fa4-8b61-a27ecaffc8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:43.076893 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a448a9-c8ef-4fa4-8b61-a27ecaffc8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:43.096295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-68a448a9-c8ef-4fa4-8b61-a27ecaffc8a1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:43.097489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-68a448a9-c8ef-4fa4-8b61-a27ecaffc8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:43.105131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:25:43.418193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7911f-97d4-4bdf-a29a-1c2a1b7f7967 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:25:43.420370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8af7911f-97d4-4bdf-a29a-1c2a1b7f7967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:43.483742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:25:43.497012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7911f-97d4-4bdf-a29a-1c2a1b7f7967 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:25:43.501147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75f71f-0472-4d0f-bbd1-0eb74856d3a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:25:43.501748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75f71f-0472-4d0f-bbd1-0eb74856d3a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:43.502060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75f71f-0472-4d0f-bbd1-0eb74856d3a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:43.502337 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75f71f-0472-4d0f-bbd1-0eb74856d3a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:43.510980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75f71f-0472-4d0f-bbd1-0eb74856d3a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:25:43.512396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75f71f-0472-4d0f-bbd1-0eb74856d3a1 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:25:43.521718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 768/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:25:45.676972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f89fa3-c02e-47f9-841c-e837277b4126 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:45.677507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f89fa3-c02e-47f9-841c-e837277b4126 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:45.677507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f89fa3-c02e-47f9-841c-e837277b4126 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:45.678887 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 05:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:25:45.684135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247933-3250-41a0-9ce3-ba38118afc22 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:45.684565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247933-3250-41a0-9ce3-ba38118afc22 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:45.684690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247933-3250-41a0-9ce3-ba38118afc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:45.704620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a9247933-3250-41a0-9ce3-ba38118afc22 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:25:45.705827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a9247933-3250-41a0-9ce3-ba38118afc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:45.713925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 05:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:25:46.017894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f89fa3-c02e-47f9-841c-e837277b4126 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:25:46.019609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b0f89fa3-c02e-47f9-841c-e837277b4126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:25:46.027115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=84 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:25:46.034136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f89fa3-c02e-47f9-841c-e837277b4126 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:25:46.037613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c7400-57bd-4789-8ea5-6e860e8e3f56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:25:46.038286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c7400-57bd-4789-8ea5-6e860e8e3f56 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:25:46.038555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c7400-57bd-4789-8ea5-6e860e8e3f56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:25:46.038829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c7400-57bd-4789-8ea5-6e860e8e3f56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:25:46.047565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c7400-57bd-4789-8ea5-6e860e8e3f56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:25:46.048971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c7400-57bd-4789-8ea5-6e860e8e3f56 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:25:46.058153 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 770/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:25:55.699448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=11 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:25:56.054104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:07.946937 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad2c2-b971-45b4-819e-f7a1179bedc1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:07.948143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad2c2-b971-45b4-819e-f7a1179bedc1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:07.948143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad2c2-b971-45b4-819e-f7a1179bedc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:07.973755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f55ad2c2-b971-45b4-819e-f7a1179bedc1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:07.975037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f55ad2c2-b971-45b4-819e-f7a1179bedc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:07.982648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca524734-36dc-4213-8607-f5fff30bfa6e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:07.983209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca524734-36dc-4213-8607-f5fff30bfa6e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:07.983209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca524734-36dc-4213-8607-f5fff30bfa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:07.987073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 770/1540] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:07.996041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499602e-2590-4844-ac7d-f25240791bf5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:07.996384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499602e-2590-4844-ac7d-f25240791bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:07.996596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499602e-2590-4844-ac7d-f25240791bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.013863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ca524734-36dc-4213-8607-f5fff30bfa6e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.018428 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ca524734-36dc-4213-8607-f5fff30bfa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.024381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c499602e-2590-4844-ac7d-f25240791bf5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.025613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c499602e-2590-4844-ac7d-f25240791bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.028069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 771/1541] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.037568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63da0f6-407c-4f9d-8b18-ed20ebc7fc5e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.039624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63da0f6-407c-4f9d-8b18-ed20ebc7fc5e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.039624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63da0f6-407c-4f9d-8b18-ed20ebc7fc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.042837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 771/1542] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:08.048516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499602e-2590-4844-ac7d-f25240791bf5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.065023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499602e-2590-4844-ac7d-f25240791bf5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.069922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa062-797a-4412-8282-8a5910ca0081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.070579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa062-797a-4412-8282-8a5910ca0081 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.070857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa062-797a-4412-8282-8a5910ca0081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.071135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa062-797a-4412-8282-8a5910ca0081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.071792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fa062-797a-4412-8282-8a5910ca0081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:08.072099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fa062-797a-4412-8282-8a5910ca0081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.074100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d63da0f6-407c-4f9d-8b18-ed20ebc7fc5e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.075429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d63da0f6-407c-4f9d-8b18-ed20ebc7fc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.078758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fa062-797a-4412-8282-8a5910ca0081 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.089234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 772/1543] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:08.093803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63da0f6-407c-4f9d-8b18-ed20ebc7fc5e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.096785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 772/1544] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:27:08.103998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63da0f6-407c-4f9d-8b18-ed20ebc7fc5e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.105661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa062-797a-4412-8282-8a5910ca0081 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.109703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.110450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.110731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.111007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.111925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:08.112228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.113252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:27:08.113252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 21 05:27:08.113252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: Sep 21 05:27:08.114154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.128593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa062-797a-4412-8282-8a5910ca0081 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.135045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 773/1545] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:27:08.145305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3322c-ca34-4618-8530-4125f11cdb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.145835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.146069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3322c-ca34-4618-8530-4125f11cdb23 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.146257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3322c-ca34-4618-8530-4125f11cdb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.146509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3322c-ca34-4618-8530-4125f11cdb23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.147241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3322c-ca34-4618-8530-4125f11cdb23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.148349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-c5e3322c-ca34-4618-8530-4125f11cdb23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:27:08.148349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Sep 21 05:27:08.148349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: Sep 21 05:27:08.149929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3322c-ca34-4618-8530-4125f11cdb23 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.156940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4a82-4893-41c3-b77d-f1dd651e8217 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.170706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316360a-b84c-4a8a-b310-fbc9b5107607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.171338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316360a-b84c-4a8a-b310-fbc9b5107607 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.171623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316360a-b84c-4a8a-b310-fbc9b5107607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.171904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316360a-b84c-4a8a-b310-fbc9b5107607 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.173112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316360a-b84c-4a8a-b310-fbc9b5107607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.177758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316360a-b84c-4a8a-b310-fbc9b5107607 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.231978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 773/1547] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:27:08.232377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 774/1547] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/projects => generated 403 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:27:08.233284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c9cad-fc16-47bd-9766-440e2c399c66 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.233580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c9cad-fc16-47bd-9766-440e2c399c66 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.233859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c9cad-fc16-47bd-9766-440e2c399c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.236330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316360a-b84c-4a8a-b310-fbc9b5107607 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.260554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-676c9cad-fc16-47bd-9766-440e2c399c66 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.261770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-676c9cad-fc16-47bd-9766-440e2c399c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.359208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316360a-b84c-4a8a-b310-fbc9b5107607 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.364478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee5941-3533-4074-878f-714a07b1a42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.365098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee5941-3533-4074-878f-714a07b1a42c admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.365380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee5941-3533-4074-878f-714a07b1a42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.365654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee5941-3533-4074-878f-714a07b1a42c admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.366318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee5941-3533-4074-878f-714a07b1a42c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.367329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-4eee5941-3533-4074-878f-714a07b1a42c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:27:08.367329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 21 05:27:08.367329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: Sep 21 05:27:08.368532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee5941-3533-4074-878f-714a07b1a42c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.394841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 774/1548] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.396549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c9cad-fc16-47bd-9766-440e2c399c66 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.406433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 775/1549] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/users => generated 302 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:27:08.409091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b16e33-3d9d-439b-9936-42fe50d823e3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.410880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b16e33-3d9d-439b-9936-42fe50d823e3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.411490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b16e33-3d9d-439b-9936-42fe50d823e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.419586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c9cad-fc16-47bd-9766-440e2c399c66 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.435933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ddf5b-1858-43c6-95ec-10ac4e6eec92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.435933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ddf5b-1858-43c6-95ec-10ac4e6eec92 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.435933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ddf5b-1858-43c6-95ec-10ac4e6eec92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.435933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ddf5b-1858-43c6-95ec-10ac4e6eec92 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.436685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ddf5b-1858-43c6-95ec-10ac4e6eec92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.438818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ddf5b-1858-43c6-95ec-10ac4e6eec92 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.446075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-03b16e33-3d9d-439b-9936-42fe50d823e3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.447279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-03b16e33-3d9d-439b-9936-42fe50d823e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.456580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 776/1550] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.459193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93558aa-1081-4895-83a6-b94442b7c8ff None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.459653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93558aa-1081-4895-83a6-b94442b7c8ff None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.460093 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93558aa-1081-4895-83a6-b94442b7c8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.462633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 775/1551] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/users => generated 292 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:08.463912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820905b9-a0bd-4089-bdac-03b67ae20c30 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.464198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820905b9-a0bd-4089-bdac-03b67ae20c30 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.464469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820905b9-a0bd-4089-bdac-03b67ae20c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.482367 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f93558aa-1081-4895-83a6-b94442b7c8ff None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.483811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f93558aa-1081-4895-83a6-b94442b7c8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.484075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-820905b9-a0bd-4089-bdac-03b67ae20c30 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.485229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-820905b9-a0bd-4089-bdac-03b67ae20c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.491909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 777/1552] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.492722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93558aa-1081-4895-83a6-b94442b7c8ff None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.493549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 776/1553] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.497873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa2c69-1783-4fcd-8e09-6016b6bee9d4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.497873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa2c69-1783-4fcd-8e09-6016b6bee9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.497873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa2c69-1783-4fcd-8e09-6016b6bee9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.503497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93558aa-1081-4895-83a6-b94442b7c8ff None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.508152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63d2a0-480c-430d-a4ae-c6f78c471989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.508755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63d2a0-480c-430d-a4ae-c6f78c471989 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.509040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63d2a0-480c-430d-a4ae-c6f78c471989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.509572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63d2a0-480c-430d-a4ae-c6f78c471989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.510262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63d2a0-480c-430d-a4ae-c6f78c471989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.511866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63d2a0-480c-430d-a4ae-c6f78c471989 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.517870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c1aa2c69-1783-4fcd-8e09-6016b6bee9d4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.519081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c1aa2c69-1783-4fcd-8e09-6016b6bee9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.525160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 778/1554] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:08.525160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade22fa4-5bff-4748-9800-6f495c967e05 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.525160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade22fa4-5bff-4748-9800-6f495c967e05 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.525160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade22fa4-5bff-4748-9800-6f495c967e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.527589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 777/1555] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.528863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2e526-5a2a-4b47-b9b4-622570c576d3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.529144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2e526-5a2a-4b47-b9b4-622570c576d3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.529517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2e526-5a2a-4b47-b9b4-622570c576d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.545391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ade22fa4-5bff-4748-9800-6f495c967e05 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.546637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ade22fa4-5bff-4748-9800-6f495c967e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.548905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fcb2e526-5a2a-4b47-b9b4-622570c576d3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.550116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fcb2e526-5a2a-4b47-b9b4-622570c576d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.554798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 779/1556] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.555974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe0bce-5f62-4110-9b3e-f38fbb4f3dca None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.556259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe0bce-5f62-4110-9b3e-f38fbb4f3dca None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.556532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe0bce-5f62-4110-9b3e-f38fbb4f3dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.564415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 778/1557] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.565212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2e526-5a2a-4b47-b9b4-622570c576d3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.574452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2e526-5a2a-4b47-b9b4-622570c576d3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.578962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c04f8-457b-4bbc-8343-fa9e18b73145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.579547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c04f8-457b-4bbc-8343-fa9e18b73145 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.579823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c04f8-457b-4bbc-8343-fa9e18b73145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.580092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c04f8-457b-4bbc-8343-fa9e18b73145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.580612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-38fe0bce-5f62-4110-9b3e-f38fbb4f3dca None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.580728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c04f8-457b-4bbc-8343-fa9e18b73145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.581808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-38fe0bce-5f62-4110-9b3e-f38fbb4f3dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.582323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c04f8-457b-4bbc-8343-fa9e18b73145 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.595318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 780/1558] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.598970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe0bce-5f62-4110-9b3e-f38fbb4f3dca None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.599066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 779/1559] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:08.599066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8ff46-63de-4c37-a4b5-d3f82fb9b022 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.599066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8ff46-63de-4c37-a4b5-d3f82fb9b022 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.599066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8ff46-63de-4c37-a4b5-d3f82fb9b022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.606209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe0bce-5f62-4110-9b3e-f38fbb4f3dca None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.610695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1dee0-3f17-4787-9b95-0e58a3c55689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.611289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1dee0-3f17-4787-9b95-0e58a3c55689 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.611823 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1dee0-3f17-4787-9b95-0e58a3c55689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.612093 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1dee0-3f17-4787-9b95-0e58a3c55689 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.612722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1dee0-3f17-4787-9b95-0e58a3c55689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:08.613022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1dee0-3f17-4787-9b95-0e58a3c55689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.614630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1dee0-3f17-4787-9b95-0e58a3c55689 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.621049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-44e8ff46-63de-4c37-a4b5-d3f82fb9b022 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.622274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-44e8ff46-63de-4c37-a4b5-d3f82fb9b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.626322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 781/1560] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:08.627091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1dee0-3f17-4787-9b95-0e58a3c55689 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.633748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 780/1561] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.635068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d99c-9b34-4b2a-ab45-a0849875982c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.635357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d99c-9b34-4b2a-ab45-a0849875982c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.635637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d99c-9b34-4b2a-ab45-a0849875982c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.639233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1dee0-3f17-4787-9b95-0e58a3c55689 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.643663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be02c5-e694-4aa0-bf08-c065316594db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.644245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be02c5-e694-4aa0-bf08-c065316594db admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.644525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be02c5-e694-4aa0-bf08-c065316594db admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.644795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be02c5-e694-4aa0-bf08-c065316594db admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.645415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be02c5-e694-4aa0-bf08-c065316594db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.646996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be02c5-e694-4aa0-bf08-c065316594db admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.655180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1484d99c-9b34-4b2a-ab45-a0849875982c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.656500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1484d99c-9b34-4b2a-ab45-a0849875982c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.657809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 782/1562] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:08.658827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf61b8-9d94-428f-abe1-c39f16d474ab None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.659107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf61b8-9d94-428f-abe1-c39f16d474ab None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.659375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf61b8-9d94-428f-abe1-c39f16d474ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.666323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 781/1563] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.668543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceea5ca-add9-45f4-bbe4-e96748e1422b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.668543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceea5ca-add9-45f4-bbe4-e96748e1422b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.668824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceea5ca-add9-45f4-bbe4-e96748e1422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.679338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8edf61b8-9d94-428f-abe1-c39f16d474ab None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.680536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8edf61b8-9d94-428f-abe1-c39f16d474ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.687487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0ceea5ca-add9-45f4-bbe4-e96748e1422b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:08.688628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 783/1564] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.688758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0ceea5ca-add9-45f4-bbe4-e96748e1422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:08.689362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf61b8-9d94-428f-abe1-c39f16d474ab None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.698664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf61b8-9d94-428f-abe1-c39f16d474ab None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.699416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 782/1565] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:08.700328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceea5ca-add9-45f4-bbe4-e96748e1422b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.706011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3691490-9796-489d-876e-48b5f29ea9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.706610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3691490-9796-489d-876e-48b5f29ea9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.706887 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3691490-9796-489d-876e-48b5f29ea9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.707161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3691490-9796-489d-876e-48b5f29ea9f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.707896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3691490-9796-489d-876e-48b5f29ea9f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:08.708089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3691490-9796-489d-876e-48b5f29ea9f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.709665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3691490-9796-489d-876e-48b5f29ea9f4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.709803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceea5ca-add9-45f4-bbe4-e96748e1422b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.714805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf5c8f-c01d-43c4-a974-453d2b1119d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.715530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf5c8f-c01d-43c4-a974-453d2b1119d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.715810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf5c8f-c01d-43c4-a974-453d2b1119d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.716078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf5c8f-c01d-43c4-a974-453d2b1119d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.716696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf5c8f-c01d-43c4-a974-453d2b1119d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.718301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf5c8f-c01d-43c4-a974-453d2b1119d7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.722014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 784/1566] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:08.722960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3691490-9796-489d-876e-48b5f29ea9f4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.731844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 783/1567] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:08.731844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf5c8f-c01d-43c4-a974-453d2b1119d7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.736126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3691490-9796-489d-876e-48b5f29ea9f4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.740863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9febdfa-5cb5-4a6a-9385-2931fa134132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.741451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9febdfa-5cb5-4a6a-9385-2931fa134132 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.741732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9febdfa-5cb5-4a6a-9385-2931fa134132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.741853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf5c8f-c01d-43c4-a974-453d2b1119d7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.742039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9febdfa-5cb5-4a6a-9385-2931fa134132 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.742690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9febdfa-5cb5-4a6a-9385-2931fa134132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.744588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9febdfa-5cb5-4a6a-9385-2931fa134132 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.746256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be853fc0-48fb-479f-a4f0-5028f9f8f090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.746852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be853fc0-48fb-479f-a4f0-5028f9f8f090 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.747127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be853fc0-48fb-479f-a4f0-5028f9f8f090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.747396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be853fc0-48fb-479f-a4f0-5028f9f8f090 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.748019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be853fc0-48fb-479f-a4f0-5028f9f8f090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:08.748316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be853fc0-48fb-479f-a4f0-5028f9f8f090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.749866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be853fc0-48fb-479f-a4f0-5028f9f8f090 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.761601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 784/1568] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:08.762383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be853fc0-48fb-479f-a4f0-5028f9f8f090 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.771917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be853fc0-48fb-479f-a4f0-5028f9f8f090 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.774621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:08.777630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f6c0e-7e8c-47e4-9a4c-491affd1064f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.778326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f6c0e-7e8c-47e4-9a4c-491affd1064f admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.778607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f6c0e-7e8c-47e4-9a4c-491affd1064f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.778867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f6c0e-7e8c-47e4-9a4c-491affd1064f admin admin] PATH_INFO: `/v3/projects/5d76fd3cc5a946d28ed3e262665de2b0/users/78a5b8b5f17d4a0fa0e558f92840a360/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.787358 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 785/1569] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:08.789065 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9febdfa-5cb5-4a6a-9385-2931fa134132 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.789191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:08.799251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f6c0e-7e8c-47e4-9a4c-491affd1064f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d76fd3cc5a946d28ed3e262665de2b0, user_id=78a5b8b5f17d4a0fa0e558f92840a360, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.800811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-760f6c0e-7e8c-47e4-9a4c-491affd1064f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:27:08.800811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Sep 21 05:27:08.800811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: Sep 21 05:27:08.803816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f6c0e-7e8c-47e4-9a4c-491affd1064f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.816999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 785/1570] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:08 2022] PUT /identity/v3/projects/5d76fd3cc5a946d28ed3e262665de2b0/users/78a5b8b5f17d4a0fa0e558f92840a360/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:08.817573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f6c0e-7e8c-47e4-9a4c-491affd1064f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.852161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9febdfa-5cb5-4a6a-9385-2931fa134132 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.885517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f6c0e-7e8c-47e4-9a4c-491affd1064f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.890261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516057b-7076-4d51-a35a-27937468492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.890856 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516057b-7076-4d51-a35a-27937468492b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.891135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516057b-7076-4d51-a35a-27937468492b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.891405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516057b-7076-4d51-a35a-27937468492b admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.892036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516057b-7076-4d51-a35a-27937468492b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:08.892335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516057b-7076-4d51-a35a-27937468492b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.893923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516057b-7076-4d51-a35a-27937468492b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.906015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 786/1571] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:08.906690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516057b-7076-4d51-a35a-27937468492b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.911852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60a624-0679-4d52-9cd3-959ce58a54f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.913101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a624-0679-4d52-9cd3-959ce58a54f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.913389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a624-0679-4d52-9cd3-959ce58a54f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.913690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a624-0679-4d52-9cd3-959ce58a54f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.914523 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60a624-0679-4d52-9cd3-959ce58a54f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:08.914832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60a624-0679-4d52-9cd3-959ce58a54f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.916084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516057b-7076-4d51-a35a-27937468492b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.916502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60a624-0679-4d52-9cd3-959ce58a54f6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.921112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb32cb7-0a44-44b0-91c1-2b9816d9b1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.921685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb32cb7-0a44-44b0-91c1-2b9816d9b1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.921983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb32cb7-0a44-44b0-91c1-2b9816d9b1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.922258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb32cb7-0a44-44b0-91c1-2b9816d9b1c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.922872 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb32cb7-0a44-44b0-91c1-2b9816d9b1c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:08.923169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb32cb7-0a44-44b0-91c1-2b9816d9b1c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.924853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb32cb7-0a44-44b0-91c1-2b9816d9b1c3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.929153 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 786/1572] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:08.930146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60a624-0679-4d52-9cd3-959ce58a54f6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.937908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 787/1573] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:08.938860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb32cb7-0a44-44b0-91c1-2b9816d9b1c3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.940722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60a624-0679-4d52-9cd3-959ce58a54f6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.945489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bab6e7-ecdc-4551-b531-71b9c4f27ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.946122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bab6e7-ecdc-4551-b531-71b9c4f27ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.946406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bab6e7-ecdc-4551-b531-71b9c4f27ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.946680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bab6e7-ecdc-4551-b531-71b9c4f27ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.947299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bab6e7-ecdc-4551-b531-71b9c4f27ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.949084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb32cb7-0a44-44b0-91c1-2b9816d9b1c3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:08.949732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bab6e7-ecdc-4551-b531-71b9c4f27ae9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.953992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a30bb-3fe5-4220-b8b4-4ac45f1e762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:08.954828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a30bb-3fe5-4220-b8b4-4ac45f1e762d admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:08.955114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a30bb-3fe5-4220-b8b4-4ac45f1e762d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:08.955419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a30bb-3fe5-4220-b8b4-4ac45f1e762d admin admin] PATH_INFO: `/v3/projects/28f31a6be8f8416f8f593eb7a0474d86/users/d3ff88c27ad24678b61bc04281a8d30f/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:08.970638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a30bb-3fe5-4220-b8b4-4ac45f1e762d admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f31a6be8f8416f8f593eb7a0474d86, user_id=d3ff88c27ad24678b61bc04281a8d30f, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:08.973368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a30bb-3fe5-4220-b8b4-4ac45f1e762d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:08.982128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 787/1574] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:08.982128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bab6e7-ecdc-4551-b531-71b9c4f27ae9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:08.986983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 788/1575] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:08 2022] PUT /identity/v3/projects/28f31a6be8f8416f8f593eb7a0474d86/users/d3ff88c27ad24678b61bc04281a8d30f/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:08.991898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a30bb-3fe5-4220-b8b4-4ac45f1e762d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.041944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bab6e7-ecdc-4551-b531-71b9c4f27ae9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.056768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a30bb-3fe5-4220-b8b4-4ac45f1e762d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.060959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:09.061568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a4c75-a3fa-4df1-b6b8-2cd3261cb97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.062203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a4c75-a3fa-4df1-b6b8-2cd3261cb97a admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.062505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a4c75-a3fa-4df1-b6b8-2cd3261cb97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.062741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a4c75-a3fa-4df1-b6b8-2cd3261cb97a admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.063417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a4c75-a3fa-4df1-b6b8-2cd3261cb97a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.065338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a4c75-a3fa-4df1-b6b8-2cd3261cb97a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.088139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 789/1576] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.092137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf1cc4-c9db-45d9-a099-b76a0897da57 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.092137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf1cc4-c9db-45d9-a099-b76a0897da57 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.092137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf1cc4-c9db-45d9-a099-b76a0897da57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.111135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8320-9bc6-412d-8f6c-ba1e0154966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.112018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8320-9bc6-412d-8f6c-ba1e0154966f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.112420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8320-9bc6-412d-8f6c-ba1e0154966f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.112704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8320-9bc6-412d-8f6c-ba1e0154966f admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.113462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d8320-9bc6-412d-8f6c-ba1e0154966f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.116788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d8320-9bc6-412d-8f6c-ba1e0154966f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.125396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-70bf1cc4-c9db-45d9-a099-b76a0897da57 None None] MFA Rules not processed for user `78a5b8b5f17d4a0fa0e558f92840a360`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:09.126640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-70bf1cc4-c9db-45d9-a099-b76a0897da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:09.144541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 788/1577] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:08 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.146849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8320-9bc6-412d-8f6c-ba1e0154966f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.147340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:09.194116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 790/1578] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:09.195068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf1cc4-c9db-45d9-a099-b76a0897da57 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.207798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf1cc4-c9db-45d9-a099-b76a0897da57 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.210194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8320-9bc6-412d-8f6c-ba1e0154966f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.212709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8093aa-6032-42bd-9de8-4376da96fe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.213915 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8093aa-6032-42bd-9de8-4376da96fe55 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.214304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8093aa-6032-42bd-9de8-4376da96fe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.214644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8093aa-6032-42bd-9de8-4376da96fe55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.215343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8093aa-6032-42bd-9de8-4376da96fe55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.215448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59278966-3ef7-4f2d-86cd-b9f3619616d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.215951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59278966-3ef7-4f2d-86cd-b9f3619616d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.216236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59278966-3ef7-4f2d-86cd-b9f3619616d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.216511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59278966-3ef7-4f2d-86cd-b9f3619616d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.217139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59278966-3ef7-4f2d-86cd-b9f3619616d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.217724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8093aa-6032-42bd-9de8-4376da96fe55 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.219068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59278966-3ef7-4f2d-86cd-b9f3619616d8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.248576 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 791/1579] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.248780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 789/1580] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.248877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8093aa-6032-42bd-9de8-4376da96fe55 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.248964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59278966-3ef7-4f2d-86cd-b9f3619616d8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.311588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8093aa-6032-42bd-9de8-4376da96fe55 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.312042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59278966-3ef7-4f2d-86cd-b9f3619616d8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.316240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cef4bd-aa03-4297-ad39-cb6695dd1189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.316665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083a578-67bf-460f-9511-5cc48ea891b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.316844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cef4bd-aa03-4297-ad39-cb6695dd1189 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.317122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cef4bd-aa03-4297-ad39-cb6695dd1189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.317263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a578-67bf-460f-9511-5cc48ea891b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.317392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cef4bd-aa03-4297-ad39-cb6695dd1189 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.317547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a578-67bf-460f-9511-5cc48ea891b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.317827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a578-67bf-460f-9511-5cc48ea891b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.318046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cef4bd-aa03-4297-ad39-cb6695dd1189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.318493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083a578-67bf-460f-9511-5cc48ea891b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.319936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cef4bd-aa03-4297-ad39-cb6695dd1189 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.320389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083a578-67bf-460f-9511-5cc48ea891b4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.342311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 790/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.343627 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30a2f7-8229-4813-bdcf-ef717ff5cd5c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.343927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30a2f7-8229-4813-bdcf-ef717ff5cd5c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.344197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30a2f7-8229-4813-bdcf-ef717ff5cd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.347319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 792/1582] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/projects => generated 403 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.348140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cef4bd-aa03-4297-ad39-cb6695dd1189 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.372171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ac30a2f7-8229-4813-bdcf-ef717ff5cd5c None None] MFA Rules not processed for user `d3ff88c27ad24678b61bc04281a8d30f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:09.373409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ac30a2f7-8229-4813-bdcf-ef717ff5cd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:09.384926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:09.411309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cef4bd-aa03-4297-ad39-cb6695dd1189 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.415861 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d801af-84db-4e54-90a9-e3ec85c6d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.416488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d801af-84db-4e54-90a9-e3ec85c6d489 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.416773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d801af-84db-4e54-90a9-e3ec85c6d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.417047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d801af-84db-4e54-90a9-e3ec85c6d489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.417684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d801af-84db-4e54-90a9-e3ec85c6d489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.419318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d801af-84db-4e54-90a9-e3ec85c6d489 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.428996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 793/1583] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.429915 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d801af-84db-4e54-90a9-e3ec85c6d489 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.439739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d801af-84db-4e54-90a9-e3ec85c6d489 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.442489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 791/1584] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:09.443278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30a2f7-8229-4813-bdcf-ef717ff5cd5c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.447261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e4565-6869-4b4e-95af-d7b23c6b52cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.448493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e4565-6869-4b4e-95af-d7b23c6b52cd admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.448894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e4565-6869-4b4e-95af-d7b23c6b52cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.449176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e4565-6869-4b4e-95af-d7b23c6b52cd admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.449829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e4565-6869-4b4e-95af-d7b23c6b52cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.451784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e4565-6869-4b4e-95af-d7b23c6b52cd admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.462977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30a2f7-8229-4813-bdcf-ef717ff5cd5c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.469039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ec568-add0-43df-9bc6-a11b07a39279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.469912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ec568-add0-43df-9bc6-a11b07a39279 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.470262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ec568-add0-43df-9bc6-a11b07a39279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.470728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ec568-add0-43df-9bc6-a11b07a39279 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.471238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ec568-add0-43df-9bc6-a11b07a39279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.473157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ec568-add0-43df-9bc6-a11b07a39279 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.474900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 794/1585] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.475533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e4565-6869-4b4e-95af-d7b23c6b52cd admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.481747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:09.485776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e4565-6869-4b4e-95af-d7b23c6b52cd admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.493023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdff6c-a468-4803-9bcf-d67d66cad0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.493685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdff6c-a468-4803-9bcf-d67d66cad0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.494185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdff6c-a468-4803-9bcf-d67d66cad0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.495396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdff6c-a468-4803-9bcf-d67d66cad0b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.495762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdff6c-a468-4803-9bcf-d67d66cad0b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.498229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdff6c-a468-4803-9bcf-d67d66cad0b1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.507271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 792/1586] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/projects => generated 405 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.508043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ec568-add0-43df-9bc6-a11b07a39279 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.527177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 795/1587] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/users => generated 286 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.528310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdff6c-a468-4803-9bcf-d67d66cad0b1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.538036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdff6c-a468-4803-9bcf-d67d66cad0b1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.573736 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ec568-add0-43df-9bc6-a11b07a39279 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.578440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e2e1c-b278-4f62-b767-d4573f04629b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.579137 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e2e1c-b278-4f62-b767-d4573f04629b admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.579486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e2e1c-b278-4f62-b767-d4573f04629b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.579768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e2e1c-b278-4f62-b767-d4573f04629b admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.580392 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e2e1c-b278-4f62-b767-d4573f04629b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.582336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e2e1c-b278-4f62-b767-d4573f04629b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.602905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e6eb7-11b6-417b-a897-df7e01f44b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.603498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e6eb7-11b6-417b-a897-df7e01f44b2f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.603771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e6eb7-11b6-417b-a897-df7e01f44b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.604097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e6eb7-11b6-417b-a897-df7e01f44b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.604675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 793/1588] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.604815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e6eb7-11b6-417b-a897-df7e01f44b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.605477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e2e1c-b278-4f62-b767-d4573f04629b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.606394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e6eb7-11b6-417b-a897-df7e01f44b2f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.615625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e2e1c-b278-4f62-b767-d4573f04629b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.617106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 796/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.617717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e6eb7-11b6-417b-a897-df7e01f44b2f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.621026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82572429-abef-44c7-8bd8-e06967237b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.621620 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572429-abef-44c7-8bd8-e06967237b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.621905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572429-abef-44c7-8bd8-e06967237b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.622213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572429-abef-44c7-8bd8-e06967237b75 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.622857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82572429-abef-44c7-8bd8-e06967237b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.624765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82572429-abef-44c7-8bd8-e06967237b75 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.627718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e6eb7-11b6-417b-a897-df7e01f44b2f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.632313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b99c8-4acb-47b2-97d1-516404dd6ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.632884 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b99c8-4acb-47b2-97d1-516404dd6ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.633157 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b99c8-4acb-47b2-97d1-516404dd6ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.633424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b99c8-4acb-47b2-97d1-516404dd6ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.634078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b99c8-4acb-47b2-97d1-516404dd6ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.636568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b99c8-4acb-47b2-97d1-516404dd6ac1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.645715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 794/1590] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/users => generated 302 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.646999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 797/1591] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.647645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b99c8-4acb-47b2-97d1-516404dd6ac1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.649927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db5f28-a2cd-4b08-b4f9-3f315060874d None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.650256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db5f28-a2cd-4b08-b4f9-3f315060874d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.650539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db5f28-a2cd-4b08-b4f9-3f315060874d None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.652266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 795/1592] 10.222.0.18 () {54 vars in 965 bytes} [Wed Sep 21 05:27:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:27:09.652808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db5f28-a2cd-4b08-b4f9-3f315060874d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.658164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b99c8-4acb-47b2-97d1-516404dd6ac1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.662749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa4f60-e235-4375-b8fe-0119d3a8ea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.663329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa4f60-e235-4375-b8fe-0119d3a8ea78 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.663620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa4f60-e235-4375-b8fe-0119d3a8ea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.663880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa4f60-e235-4375-b8fe-0119d3a8ea78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.664500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa4f60-e235-4375-b8fe-0119d3a8ea78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.666130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa4f60-e235-4375-b8fe-0119d3a8ea78 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.667143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db5f28-a2cd-4b08-b4f9-3f315060874d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.671668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3fbb7-4338-4452-9830-178e5e79ff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.672255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3fbb7-4338-4452-9830-178e5e79ff1d admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.672540 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3fbb7-4338-4452-9830-178e5e79ff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.672814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3fbb7-4338-4452-9830-178e5e79ff1d admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.673438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3fbb7-4338-4452-9830-178e5e79ff1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.675366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3fbb7-4338-4452-9830-178e5e79ff1d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.676875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 798/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.677652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa4f60-e235-4375-b8fe-0119d3a8ea78 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.687961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa4f60-e235-4375-b8fe-0119d3a8ea78 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.693434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1045ea-fd24-48b6-836c-ca8e614d6331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.694045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1045ea-fd24-48b6-836c-ca8e614d6331 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.694322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1045ea-fd24-48b6-836c-ca8e614d6331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.694589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1045ea-fd24-48b6-836c-ca8e614d6331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.695210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1045ea-fd24-48b6-836c-ca8e614d6331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.696935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1045ea-fd24-48b6-836c-ca8e614d6331 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.698033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 796/1594] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/users => generated 309 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:09.698846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3fbb7-4338-4452-9830-178e5e79ff1d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.708544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3fbb7-4338-4452-9830-178e5e79ff1d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.710956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 799/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.711638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1045ea-fd24-48b6-836c-ca8e614d6331 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.713796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6cadd-f84d-4a53-a14a-50b78e5b1190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.714619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6cadd-f84d-4a53-a14a-50b78e5b1190 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.715048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6cadd-f84d-4a53-a14a-50b78e5b1190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.715350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6cadd-f84d-4a53-a14a-50b78e5b1190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.715988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6cadd-f84d-4a53-a14a-50b78e5b1190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.717653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6cadd-f84d-4a53-a14a-50b78e5b1190 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.721227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1045ea-fd24-48b6-836c-ca8e614d6331 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.726018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ae0e1-9497-44a1-b7f7-cf33226c2c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.726624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ae0e1-9497-44a1-b7f7-cf33226c2c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.726903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ae0e1-9497-44a1-b7f7-cf33226c2c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.727188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ae0e1-9497-44a1-b7f7-cf33226c2c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.727807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ae0e1-9497-44a1-b7f7-cf33226c2c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.731468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 797/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.731468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6cadd-f84d-4a53-a14a-50b78e5b1190 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.731636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ae0e1-9497-44a1-b7f7-cf33226c2c78 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.739891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6cadd-f84d-4a53-a14a-50b78e5b1190 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.740614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 800/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.741406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ae0e1-9497-44a1-b7f7-cf33226c2c78 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.747399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfcc5f-f99a-482b-ac7f-fe2533b80504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.747978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfcc5f-f99a-482b-ac7f-fe2533b80504 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.748261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfcc5f-f99a-482b-ac7f-fe2533b80504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.748551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfcc5f-f99a-482b-ac7f-fe2533b80504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.749183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfcc5f-f99a-482b-ac7f-fe2533b80504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.750805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfcc5f-f99a-482b-ac7f-fe2533b80504 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.751342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ae0e1-9497-44a1-b7f7-cf33226c2c78 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.756699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaca51-44f2-4ef3-a0f7-189dcceee657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.757380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaca51-44f2-4ef3-a0f7-189dcceee657 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.757598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaca51-44f2-4ef3-a0f7-189dcceee657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.758049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaca51-44f2-4ef3-a0f7-189dcceee657 admin admin] PATH_INFO: `/v3/projects/e8fac41f41e5499aae33fd726b61f8a7/users/e7527106829643448f153f9431f233f1/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.761612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 798/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.762128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae02ec7-8866-44e1-9855-3012d2930d6b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.762416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae02ec7-8866-44e1-9855-3012d2930d6b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.762691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae02ec7-8866-44e1-9855-3012d2930d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.772326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaca51-44f2-4ef3-a0f7-189dcceee657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8fac41f41e5499aae33fd726b61f8a7, user_id=e7527106829643448f153f9431f233f1, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.774993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaca51-44f2-4ef3-a0f7-189dcceee657 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.788007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 801/1599] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:09 2022] PUT /identity/v3/projects/e8fac41f41e5499aae33fd726b61f8a7/users/e7527106829643448f153f9431f233f1/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:09.793262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaca51-44f2-4ef3-a0f7-189dcceee657 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.800803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0ae02ec7-8866-44e1-9855-3012d2930d6b None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:09.802145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0ae02ec7-8866-44e1-9855-3012d2930d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:09.808087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:09.855813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaca51-44f2-4ef3-a0f7-189dcceee657 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.861036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9feed56-cce3-4cbc-8b42-0dad1258090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.861991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9feed56-cce3-4cbc-8b42-0dad1258090d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.862299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9feed56-cce3-4cbc-8b42-0dad1258090d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.862575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9feed56-cce3-4cbc-8b42-0dad1258090d admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.863228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9feed56-cce3-4cbc-8b42-0dad1258090d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.864805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9feed56-cce3-4cbc-8b42-0dad1258090d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.872771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 799/1600] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:09.873602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae02ec7-8866-44e1-9855-3012d2930d6b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.878568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 802/1601] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.878568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9feed56-cce3-4cbc-8b42-0dad1258090d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.883774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae02ec7-8866-44e1-9855-3012d2930d6b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.887607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9feed56-cce3-4cbc-8b42-0dad1258090d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.888349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea135f3-ebff-403b-9b9b-e165469b4935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.888948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea135f3-ebff-403b-9b9b-e165469b4935 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.889237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea135f3-ebff-403b-9b9b-e165469b4935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.889518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea135f3-ebff-403b-9b9b-e165469b4935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.890579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea135f3-ebff-403b-9b9b-e165469b4935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.892300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02284bf-3627-46e8-861d-2f8b35eab0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.892885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02284bf-3627-46e8-861d-2f8b35eab0aa admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.893178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02284bf-3627-46e8-861d-2f8b35eab0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.893430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02284bf-3627-46e8-861d-2f8b35eab0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.893523 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea135f3-ebff-403b-9b9b-e165469b4935 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.894088 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02284bf-3627-46e8-861d-2f8b35eab0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.895680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02284bf-3627-46e8-861d-2f8b35eab0aa admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.901728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:09.904495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 800/1602] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.905272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea135f3-ebff-403b-9b9b-e165469b4935 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.910760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 803/1603] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.911592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02284bf-3627-46e8-861d-2f8b35eab0aa admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.915687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea135f3-ebff-403b-9b9b-e165469b4935 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.920861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fad22c-ad27-401a-b0a2-aed6cea3849f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.921453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fad22c-ad27-401a-b0a2-aed6cea3849f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.921732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fad22c-ad27-401a-b0a2-aed6cea3849f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.922037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fad22c-ad27-401a-b0a2-aed6cea3849f admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.922667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fad22c-ad27-401a-b0a2-aed6cea3849f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.923209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02284bf-3627-46e8-861d-2f8b35eab0aa admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.924250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fad22c-ad27-401a-b0a2-aed6cea3849f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.927775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c6ba5-db12-48c7-a18e-ecca0524b083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.928444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c6ba5-db12-48c7-a18e-ecca0524b083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.928724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c6ba5-db12-48c7-a18e-ecca0524b083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.929000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c6ba5-db12-48c7-a18e-ecca0524b083 admin admin] PATH_INFO: `/v3/projects/58c4b56493c94c2fb8e5e595cef89a55/users/9fb2f0174dd04c0fb2aa0ac142ffce95/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.937809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 801/1604] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.937809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fad22c-ad27-401a-b0a2-aed6cea3849f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.944610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c6ba5-db12-48c7-a18e-ecca0524b083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c4b56493c94c2fb8e5e595cef89a55, user_id=9fb2f0174dd04c0fb2aa0ac142ffce95, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.945002 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fad22c-ad27-401a-b0a2-aed6cea3849f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.947322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c6ba5-db12-48c7-a18e-ecca0524b083 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.949559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8627175-8914-46de-ba7f-370fe8b39cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:09.950163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8627175-8914-46de-ba7f-370fe8b39cad admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.950444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8627175-8914-46de-ba7f-370fe8b39cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.950715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8627175-8914-46de-ba7f-370fe8b39cad admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.951338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8627175-8914-46de-ba7f-370fe8b39cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:09.953138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8627175-8914-46de-ba7f-370fe8b39cad admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:09.960950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 804/1605] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:09 2022] PUT /identity/v3/projects/58c4b56493c94c2fb8e5e595cef89a55/users/9fb2f0174dd04c0fb2aa0ac142ffce95/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:09.962783 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8cecc-7504-42ad-991f-4961a2c6d0dd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:09.963479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8cecc-7504-42ad-991f-4961a2c6d0dd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:09.964201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 802/1606] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:09.964445 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8cecc-7504-42ad-991f-4961a2c6d0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:09.965054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8627175-8914-46de-ba7f-370fe8b39cad admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:09.975658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8627175-8914-46de-ba7f-370fe8b39cad admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:09.984488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c7b8cecc-7504-42ad-991f-4961a2c6d0dd None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:09.985678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c7b8cecc-7504-42ad-991f-4961a2c6d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:10.031553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2e134-b84d-4aad-a75b-24f8815e5aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.032128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2e134-b84d-4aad-a75b-24f8815e5aad admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.032405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2e134-b84d-4aad-a75b-24f8815e5aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.032751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2e134-b84d-4aad-a75b-24f8815e5aad admin admin] PATH_INFO: `/v3/projects/d9288b120436428e86b5ff4e87f5d349/users/75d4379ec87240389ce9c077886ab472/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.045031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 805/1607] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:10.046105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8cecc-7504-42ad-991f-4961a2c6d0dd None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.048000 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2e134-b84d-4aad-a75b-24f8815e5aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9288b120436428e86b5ff4e87f5d349, user_id=75d4379ec87240389ce9c077886ab472, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.051673 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2e134-b84d-4aad-a75b-24f8815e5aad admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.065167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 803/1608] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:09 2022] PUT /identity/v3/projects/d9288b120436428e86b5ff4e87f5d349/users/75d4379ec87240389ce9c077886ab472/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:10.065973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2e134-b84d-4aad-a75b-24f8815e5aad admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.117101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:10.118854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8cecc-7504-42ad-991f-4961a2c6d0dd None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.128562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2e134-b84d-4aad-a75b-24f8815e5aad admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.133230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffe218-1d71-438b-b364-8dd1b47619cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.133883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffe218-1d71-438b-b364-8dd1b47619cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.134188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffe218-1d71-438b-b364-8dd1b47619cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.134458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffe218-1d71-438b-b364-8dd1b47619cb admin admin] PATH_INFO: `/v3/projects/415cd9ddca8f406d898b1badb80fdb90/users/03005a75121a4dc79db619c81c26168c/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.148793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffe218-1d71-438b-b364-8dd1b47619cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=415cd9ddca8f406d898b1badb80fdb90, user_id=03005a75121a4dc79db619c81c26168c, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.152315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffe218-1d71-438b-b364-8dd1b47619cb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.165272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 804/1609] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:10 2022] PUT /identity/v3/projects/415cd9ddca8f406d898b1badb80fdb90/users/03005a75121a4dc79db619c81c26168c/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:10.166159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffe218-1d71-438b-b364-8dd1b47619cb admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.190108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bbe39-0c64-4a74-91bd-a90bfc5eed5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.191103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bbe39-0c64-4a74-91bd-a90bfc5eed5a service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.191386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bbe39-0c64-4a74-91bd-a90bfc5eed5a service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.191657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bbe39-0c64-4a74-91bd-a90bfc5eed5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.201318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:10.227862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffe218-1d71-438b-b364-8dd1b47619cb admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.232987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a41d48-5bf1-438c-a293-5aa0125b2d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.233638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41d48-5bf1-438c-a293-5aa0125b2d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.233916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41d48-5bf1-438c-a293-5aa0125b2d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.234216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41d48-5bf1-438c-a293-5aa0125b2d34 admin admin] PATH_INFO: `/v3/projects/91291204c34f46449d96ca93d93672e4/users/c8ccedc126aa487f8fdfb50ddfd1b264/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.249519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a41d48-5bf1-438c-a293-5aa0125b2d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91291204c34f46449d96ca93d93672e4, user_id=c8ccedc126aa487f8fdfb50ddfd1b264, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.251139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537bbe39-0c64-4a74-91bd-a90bfc5eed5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.252330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a41d48-5bf1-438c-a293-5aa0125b2d34 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.252757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537bbe39-0c64-4a74-91bd-a90bfc5eed5a service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.262176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 806/1610] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:10.263013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bbe39-0c64-4a74-91bd-a90bfc5eed5a service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.277394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 805/1611] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:10 2022] PUT /identity/v3/projects/91291204c34f46449d96ca93d93672e4/users/c8ccedc126aa487f8fdfb50ddfd1b264/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:10.277394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8602e-eb75-4728-840c-1290c68234e0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.277394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8602e-eb75-4728-840c-1290c68234e0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.277394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8602e-eb75-4728-840c-1290c68234e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.285220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bbe39-0c64-4a74-91bd-a90bfc5eed5a service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.291214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b3b8602e-eb75-4728-840c-1290c68234e0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:10.292979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b3b8602e-eb75-4728-840c-1290c68234e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:10.348128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbf421-7da0-4f16-8927-ae9885449b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.348835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf421-7da0-4f16-8927-ae9885449b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.349232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf421-7da0-4f16-8927-ae9885449b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.349513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf421-7da0-4f16-8927-ae9885449b1b admin admin] PATH_INFO: `/v3/projects/53b1fb5bb9cd41959ebca2804fb426a2/users/1ec24729b04947b7b05cc6dcd787eb9a/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.358911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 806/1612] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:10.359679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8602e-eb75-4728-840c-1290c68234e0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.368798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbf421-7da0-4f16-8927-ae9885449b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b1fb5bb9cd41959ebca2804fb426a2, user_id=1ec24729b04947b7b05cc6dcd787eb9a, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.369861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8602e-eb75-4728-840c-1290c68234e0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.371796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbf421-7da0-4f16-8927-ae9885449b1b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.375165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dadca9-3733-40fc-a6a8-aa469c9c246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.375610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dadca9-3733-40fc-a6a8-aa469c9c246c admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.375891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dadca9-3733-40fc-a6a8-aa469c9c246c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.376168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dadca9-3733-40fc-a6a8-aa469c9c246c admin admin] PATH_INFO: `/v3/projects/50c624b022e8481f8ce19587e1149b54/users/035b773f4fec49babaf6418272cfde73/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.388185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 807/1613] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:10 2022] PUT /identity/v3/projects/53b1fb5bb9cd41959ebca2804fb426a2/users/1ec24729b04947b7b05cc6dcd787eb9a/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:10.390099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbf421-7da0-4f16-8927-ae9885449b1b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.393229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dadca9-3733-40fc-a6a8-aa469c9c246c admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c624b022e8481f8ce19587e1149b54, user_id=035b773f4fec49babaf6418272cfde73, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.396099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dadca9-3733-40fc-a6a8-aa469c9c246c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.412932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 807/1614] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:10 2022] PUT /identity/v3/projects/50c624b022e8481f8ce19587e1149b54/users/035b773f4fec49babaf6418272cfde73/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:10.417459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b266c-6e80-4813-8b20-f1752c03a5fb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.417459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b266c-6e80-4813-8b20-f1752c03a5fb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.417459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b266c-6e80-4813-8b20-f1752c03a5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.444818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-da8b266c-6e80-4813-8b20-f1752c03a5fb None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:10.446058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-da8b266c-6e80-4813-8b20-f1752c03a5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:10.458507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbf421-7da0-4f16-8927-ae9885449b1b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.478545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:10.512446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e021959-adab-4976-a9ca-db00a51fb1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.513057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e021959-adab-4976-a9ca-db00a51fb1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.513370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e021959-adab-4976-a9ca-db00a51fb1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.513489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 808/1615] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:10.513663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e021959-adab-4976-a9ca-db00a51fb1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.517628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b266c-6e80-4813-8b20-f1752c03a5fb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.519717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:10.524175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e021959-adab-4976-a9ca-db00a51fb1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.527097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e021959-adab-4976-a9ca-db00a51fb1d5 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.527905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b266c-6e80-4813-8b20-f1752c03a5fb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.532492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efefb50-8e52-47f7-a282-38f3c2aaa82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.533122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efefb50-8e52-47f7-a282-38f3c2aaa82f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.533402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efefb50-8e52-47f7-a282-38f3c2aaa82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.533677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efefb50-8e52-47f7-a282-38f3c2aaa82f admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.534348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efefb50-8e52-47f7-a282-38f3c2aaa82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.535986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efefb50-8e52-47f7-a282-38f3c2aaa82f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.537695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 808/1616] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:10.538982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e575ef-0184-4e04-9ae6-8e05d9d7a241 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.539265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e575ef-0184-4e04-9ae6-8e05d9d7a241 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.539542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e575ef-0184-4e04-9ae6-8e05d9d7a241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.549131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 809/1617] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:10.549131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efefb50-8e52-47f7-a282-38f3c2aaa82f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.557609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efefb50-8e52-47f7-a282-38f3c2aaa82f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.561881 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba4170-379b-4f1d-ac27-ce9898f3b181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.562532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba4170-379b-4f1d-ac27-ce9898f3b181 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.563502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba4170-379b-4f1d-ac27-ce9898f3b181 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.563502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba4170-379b-4f1d-ac27-ce9898f3b181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.570779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-47e575ef-0184-4e04-9ae6-8e05d9d7a241 None None] MFA Rules not processed for user `c8ccedc126aa487f8fdfb50ddfd1b264`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:10.571986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-47e575ef-0184-4e04-9ae6-8e05d9d7a241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:10.574108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba4170-379b-4f1d-ac27-ce9898f3b181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.575371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba4170-379b-4f1d-ac27-ce9898f3b181 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.590815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 810/1618] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:10.592916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a064b-8bd3-4815-94b2-c689946687b1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.592916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a064b-8bd3-4815-94b2-c689946687b1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.593138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a064b-8bd3-4815-94b2-c689946687b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.625271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6a0a064b-8bd3-4815-94b2-c689946687b1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:10.626842 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6a0a064b-8bd3-4815-94b2-c689946687b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:10.641154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 811/1619] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:10.642247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f29de4-4851-449e-a806-1f57b3a1d7f2 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.642537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f29de4-4851-449e-a806-1f57b3a1d7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.642808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f29de4-4851-449e-a806-1f57b3a1d7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.659980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 809/1620] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:10.661551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7601ed-df85-4546-89ab-5cf7eb8dae86 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.661908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7601ed-df85-4546-89ab-5cf7eb8dae86 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.662539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7601ed-df85-4546-89ab-5cf7eb8dae86 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.664532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 810/1621] 10.222.0.18 () {54 vars in 912 bytes} [Wed Sep 21 05:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:27:10.665366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7601ed-df85-4546-89ab-5cf7eb8dae86 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.668434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-56f29de4-4851-449e-a806-1f57b3a1d7f2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:10.669641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-56f29de4-4851-449e-a806-1f57b3a1d7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:10.679197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7601ed-df85-4546-89ab-5cf7eb8dae86 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.681273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 812/1622] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:10.682369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f29de4-4851-449e-a806-1f57b3a1d7f2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.684422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe26abd-fef8-483c-8f96-117d728b0166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.685204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe26abd-fef8-483c-8f96-117d728b0166 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.685486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe26abd-fef8-483c-8f96-117d728b0166 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.685827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe26abd-fef8-483c-8f96-117d728b0166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.693009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f29de4-4851-449e-a806-1f57b3a1d7f2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.700209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef2d48-3e5f-4672-9930-b62e35316889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.701129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef2d48-3e5f-4672-9930-b62e35316889 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.701542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef2d48-3e5f-4672-9930-b62e35316889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.701654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe26abd-fef8-483c-8f96-117d728b0166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.701794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef2d48-3e5f-4672-9930-b62e35316889 admin admin] PATH_INFO: `/v3/projects/415cd9ddca8f406d898b1badb80fdb90/users/03005a75121a4dc79db619c81c26168c/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.703803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe26abd-fef8-483c-8f96-117d728b0166 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.704208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef2d48-3e5f-4672-9930-b62e35316889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=415cd9ddca8f406d898b1badb80fdb90, user_id=03005a75121a4dc79db619c81c26168c, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.707619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef2d48-3e5f-4672-9930-b62e35316889 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.717185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 811/1623] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:10.718624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe26abd-fef8-483c-8f96-117d728b0166 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.723335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 813/1624] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:10 2022] PUT /identity/v3/projects/415cd9ddca8f406d898b1badb80fdb90/users/03005a75121a4dc79db619c81c26168c/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:10.723335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fc78e-44f7-4b88-8476-8560a2895c7b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.723335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fc78e-44f7-4b88-8476-8560a2895c7b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.726483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fc78e-44f7-4b88-8476-8560a2895c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.740698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe26abd-fef8-483c-8f96-117d728b0166 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.758813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-755fc78e-44f7-4b88-8476-8560a2895c7b None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:10.760750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-755fc78e-44f7-4b88-8476-8560a2895c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:10.809254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93451-acc4-49ae-91c6-0227e83a598b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.810087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93451-acc4-49ae-91c6-0227e83a598b service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.810256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93451-acc4-49ae-91c6-0227e83a598b service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.810597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93451-acc4-49ae-91c6-0227e83a598b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.843591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 814/1625] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:27:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:10.844453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fc78e-44f7-4b88-8476-8560a2895c7b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:10.853991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fc78e-44f7-4b88-8476-8560a2895c7b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:10.858572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8382c-a579-43fe-a904-8695cff963a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:10.859203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8382c-a579-43fe-a904-8695cff963a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.859616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8382c-a579-43fe-a904-8695cff963a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.859887 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8382c-a579-43fe-a904-8695cff963a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.886463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e93451-acc4-49ae-91c6-0227e83a598b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.888076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e93451-acc4-49ae-91c6-0227e83a598b service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.901236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 812/1626] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:10.902776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f587b-3c3b-4233-861d-0adb62b5eba8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.903066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f587b-3c3b-4233-861d-0adb62b5eba8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.903344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f587b-3c3b-4233-861d-0adb62b5eba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.941623 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8b5f587b-3c3b-4233-861d-0adb62b5eba8 None None] MFA Rules not processed for user `03005a75121a4dc79db619c81c26168c`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:10.944646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8b5f587b-3c3b-4233-861d-0adb62b5eba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:10.955601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:10.969731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8382c-a579-43fe-a904-8695cff963a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:10.971398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8382c-a579-43fe-a904-8695cff963a0 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:10.987182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 815/1627] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:10.988690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b57581-529f-448c-8329-f25aa497c81e None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:10.988820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b57581-529f-448c-8329-f25aa497c81e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:10.989539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b57581-529f-448c-8329-f25aa497c81e None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:10.991452 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 816/1628] 10.222.0.18 () {54 vars in 965 bytes} [Wed Sep 21 05:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:27:10.992655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b57581-529f-448c-8329-f25aa497c81e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:11.014618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:11.042875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 813/1629] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:10 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:11.044349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053325b-ca9b-4ec2-85c6-8f1e2c554a4d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:11.044674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053325b-ca9b-4ec2-85c6-8f1e2c554a4d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:11.045173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053325b-ca9b-4ec2-85c6-8f1e2c554a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:11.069616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5053325b-ca9b-4ec2-85c6-8f1e2c554a4d None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:11.070886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5053325b-ca9b-4ec2-85c6-8f1e2c554a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:11.084885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 814/1630] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Sep 21 05:27:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:11.095444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053325b-ca9b-4ec2-85c6-8f1e2c554a4d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:11.097557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b57581-529f-448c-8329-f25aa497c81e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:11.102360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e6551-5252-4c2c-be89-16eac930f06f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:11.103185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e6551-5252-4c2c-be89-16eac930f06f service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:11.103529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e6551-5252-4c2c-be89-16eac930f06f service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:11.103948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e6551-5252-4c2c-be89-16eac930f06f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:11.105265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053325b-ca9b-4ec2-85c6-8f1e2c554a4d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:11.109989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c6edb-9f3a-4a20-ae06-4df24004b74d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:11.110610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c6edb-9f3a-4a20-ae06-4df24004b74d service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:11.110890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c6edb-9f3a-4a20-ae06-4df24004b74d service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:11.111163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c6edb-9f3a-4a20-ae06-4df24004b74d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:11.124604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c6edb-9f3a-4a20-ae06-4df24004b74d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:11.124759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e6551-5252-4c2c-be89-16eac930f06f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:11.126153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c6edb-9f3a-4a20-ae06-4df24004b74d service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:11.126726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e6551-5252-4c2c-be89-16eac930f06f service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:11.142037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 815/1631] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:11 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:11.142732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c6edb-9f3a-4a20-ae06-4df24004b74d service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:11.158063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c6edb-9f3a-4a20-ae06-4df24004b74d service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:11.165900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4402b-e1e9-4017-a132-f9a2c30a2773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:11.167156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4402b-e1e9-4017-a132-f9a2c30a2773 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:11.167443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4402b-e1e9-4017-a132-f9a2c30a2773 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:11.167720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4402b-e1e9-4017-a132-f9a2c30a2773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:11.169216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 817/1632] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:11.275496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4402b-e1e9-4017-a132-f9a2c30a2773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:11.277342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4402b-e1e9-4017-a132-f9a2c30a2773 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:11.290730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 816/1633] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:11.399454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e6551-5252-4c2c-be89-16eac930f06f service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:11.424415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e6551-5252-4c2c-be89-16eac930f06f service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:11.427195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:11.428643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:11.429183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:11.429687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] PATH_INFO: `/v3/users/78a5b8b5f17d4a0fa0e558f92840a360` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:11.431238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a5b8b5f17d4a0fa0e558f92840a360)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:11.434121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:11.504631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a5b8b5f17d4a0fa0e558f92840a360'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:11.527682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a5b8b5f17d4a0fa0e558f92840a360'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:11.528998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a5b8b5f17d4a0fa0e558f92840a360'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:11.529794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a5b8b5f17d4a0fa0e558f92840a360'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:11.543790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8c6133b7-5009-4943-90af-8021293c21f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a5b8b5f17d4a0fa0e558f92840a360'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:11.590535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 818/1634] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:11 2022] DELETE /identity/v3/users/78a5b8b5f17d4a0fa0e558f92840a360 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:11.595672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c4b82-f735-448f-adf8-dde957e0f9ac None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:11.595956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c4b82-f735-448f-adf8-dde957e0f9ac None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:11.596231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c4b82-f735-448f-adf8-dde957e0f9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:11.631477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-020c4b82-f735-448f-adf8-dde957e0f9ac None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:11.635020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-020c4b82-f735-448f-adf8-dde957e0f9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:11.644202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ba6cd-d180-49c2-9d5c-aa676488172e None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:11.644689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ba6cd-d180-49c2-9d5c-aa676488172e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:11.644817 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ba6cd-d180-49c2-9d5c-aa676488172e None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:11.646878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 819/1635] 10.222.0.18 () {54 vars in 912 bytes} [Wed Sep 21 05:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:27:11.654321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f444ba-94db-4d5b-9ff2-81171d060d79 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:11.654636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f444ba-94db-4d5b-9ff2-81171d060d79 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:11.654893 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f444ba-94db-4d5b-9ff2-81171d060d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:11.697343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e7f444ba-94db-4d5b-9ff2-81171d060d79 None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:11.702030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e7f444ba-94db-4d5b-9ff2-81171d060d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:11.729438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 817/1636] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:11.747519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c4b82-f735-448f-adf8-dde957e0f9ac None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:11.750217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-020c4b82-f735-448f-adf8-dde957e0f9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:11.812505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:11.871764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 820/1637] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:27:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:11.874800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c4b82-f735-448f-adf8-dde957e0f9ac None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:11.879569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d37e3-d23e-4424-b2c1-8c4137a95cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:11.880199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d37e3-d23e-4424-b2c1-8c4137a95cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:11.880525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d37e3-d23e-4424-b2c1-8c4137a95cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:11.880810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d37e3-d23e-4424-b2c1-8c4137a95cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:11.887821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f444ba-94db-4d5b-9ff2-81171d060d79 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:11.889406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e7f444ba-94db-4d5b-9ff2-81171d060d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:11.909467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d37e3-d23e-4424-b2c1-8c4137a95cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:11.911377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d37e3-d23e-4424-b2c1-8c4137a95cfb service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:11.927621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:11.941192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 818/1638] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:27:12.037902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f444ba-94db-4d5b-9ff2-81171d060d79 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:12.055909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6c483-81cd-4409-9c94-1a60886a84ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:12.058887 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6c483-81cd-4409-9c94-1a60886a84ae service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:12.059571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6c483-81cd-4409-9c94-1a60886a84ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:12.060104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6c483-81cd-4409-9c94-1a60886a84ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:12.095619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6c483-81cd-4409-9c94-1a60886a84ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:12.097188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6c483-81cd-4409-9c94-1a60886a84ae service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:12.124754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 821/1639] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:27:12.965061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119398ab-20e3-48f8-a7b1-837391d62f6b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:12.966284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119398ab-20e3-48f8-a7b1-837391d62f6b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:12.966787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119398ab-20e3-48f8-a7b1-837391d62f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:13.019798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-119398ab-20e3-48f8-a7b1-837391d62f6b None None] MFA Rules not processed for user `035b773f4fec49babaf6418272cfde73`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:13.022316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-119398ab-20e3-48f8-a7b1-837391d62f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:13.145854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 819/1640] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:12 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:13.150197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6c483-81cd-4409-9c94-1a60886a84ae service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:13.152843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-83d6c483-81cd-4409-9c94-1a60886a84ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:13.173274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6c483-81cd-4409-9c94-1a60886a84ae service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:13.181212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0329b-a961-4d22-8c53-0766c381ca60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:13.182530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0329b-a961-4d22-8c53-0766c381ca60 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:13.183077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0329b-a961-4d22-8c53-0766c381ca60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:13.183564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0329b-a961-4d22-8c53-0766c381ca60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:13.184661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0329b-a961-4d22-8c53-0766c381ca60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:13.187212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0329b-a961-4d22-8c53-0766c381ca60 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:13.201058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119398ab-20e3-48f8-a7b1-837391d62f6b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:13.202797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-119398ab-20e3-48f8-a7b1-837391d62f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:13.225265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119398ab-20e3-48f8-a7b1-837391d62f6b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:13.240080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:13.242910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 822/1641] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:13 2022] POST /identity/v3/projects => generated 403 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:13.243462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:13.244161 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:13.244697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] PATH_INFO: `/v3/projects/5d76fd3cc5a946d28ed3e262665de2b0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:13.246363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d76fd3cc5a946d28ed3e262665de2b0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:13.247660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0329b-a961-4d22-8c53-0766c381ca60 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:13.250368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:13.351845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d76fd3cc5a946d28ed3e262665de2b0', 'request_id': 'req-9b204410-bea0-48b2-979b-beb8e82b5aac'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.381031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d76fd3cc5a946d28ed3e262665de2b0', 'request_id': 'req-9b204410-bea0-48b2-979b-beb8e82b5aac'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.389976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0329b-a961-4d22-8c53-0766c381ca60 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:13.399304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d76fd3cc5a946d28ed3e262665de2b0', 'request_id': 'req-9b204410-bea0-48b2-979b-beb8e82b5aac'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.420709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] The token cache is being invalidate because project 5d76fd3cc5a946d28ed3e262665de2b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:13.422145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.424536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.431600 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 820/1642] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:13 2022] DELETE /identity/v3/projects/5d76fd3cc5a946d28ed3e262665de2b0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:27:13.432908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:13.435925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:13.480811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7927c-347d-4d6a-baa2-9844a14f3f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:13.481447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7927c-347d-4d6a-baa2-9844a14f3f81 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:13.481730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7927c-347d-4d6a-baa2-9844a14f3f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:13.482035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7927c-347d-4d6a-baa2-9844a14f3f81 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:13.482673 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7927c-347d-4d6a-baa2-9844a14f3f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:13.484579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7927c-347d-4d6a-baa2-9844a14f3f81 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:13.535741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 823/1643] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:13 2022] POST /identity/v3/users => generated 308 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:13.536713 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7927c-347d-4d6a-baa2-9844a14f3f81 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:13.544121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bbb7927c-347d-4d6a-baa2-9844a14f3f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:13.556672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:13.559651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7927c-347d-4d6a-baa2-9844a14f3f81 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:13.568184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c52f0-8c4f-43df-95ce-571f94f9a1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:13.568975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c52f0-8c4f-43df-95ce-571f94f9a1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:13.570072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c52f0-8c4f-43df-95ce-571f94f9a1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:13.570521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c52f0-8c4f-43df-95ce-571f94f9a1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:13.571481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c52f0-8c4f-43df-95ce-571f94f9a1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:13.573856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c52f0-8c4f-43df-95ce-571f94f9a1f4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:13.592649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 824/1644] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:13.592649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c52f0-8c4f-43df-95ce-571f94f9a1f4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:13.604584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c52f0-8c4f-43df-95ce-571f94f9a1f4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:13.609112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b204410-bea0-48b2-979b-beb8e82b5aac admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:13.610031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6647d-627f-4f72-910c-206818a729b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:13.611953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6647d-627f-4f72-910c-206818a729b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:13.612545 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6647d-627f-4f72-910c-206818a729b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:13.612837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6647d-627f-4f72-910c-206818a729b0 admin admin] PATH_INFO: `/v3/projects/343afb1e5ab3446eb16ba680491e692e/users/0b37f08b619f45229f55f27f7682a9c2/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:13.621722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:13.622710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:13.624078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:13.624469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] PATH_INFO: `/v3/users/03005a75121a4dc79db619c81c26168c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:13.627560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03005a75121a4dc79db619c81c26168c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:13.629536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:27:13.629536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Sep 21 05:27:13.629536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: Sep 21 05:27:13.633841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:13.657042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6647d-627f-4f72-910c-206818a729b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=343afb1e5ab3446eb16ba680491e692e, user_id=0b37f08b619f45229f55f27f7682a9c2, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:13.661829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6647d-627f-4f72-910c-206818a729b0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:13.684526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:13.692833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 825/1645] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:13 2022] PUT /identity/v3/projects/343afb1e5ab3446eb16ba680491e692e/users/0b37f08b619f45229f55f27f7682a9c2/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:13.700388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6647d-627f-4f72-910c-206818a729b0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:13.818843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03005a75121a4dc79db619c81c26168c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.821920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03005a75121a4dc79db619c81c26168c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.840319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03005a75121a4dc79db619c81c26168c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.841017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03005a75121a4dc79db619c81c26168c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.865366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03005a75121a4dc79db619c81c26168c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:13.881610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6647d-627f-4f72-910c-206818a729b0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:13.894119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a46ad-8a1b-468f-9372-5710d2042668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:13.896651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a46ad-8a1b-468f-9372-5710d2042668 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:13.896910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a46ad-8a1b-468f-9372-5710d2042668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:13.897195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a46ad-8a1b-468f-9372-5710d2042668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:13.898372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a46ad-8a1b-468f-9372-5710d2042668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:13.903540 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a46ad-8a1b-468f-9372-5710d2042668 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:13.927728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 821/1646] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:13 2022] DELETE /identity/v3/users/03005a75121a4dc79db619c81c26168c => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:13.931057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:13.932535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 826/1647] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:13.933267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:14.039746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1c25c-edbf-452c-a43c-c397bda41224 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:14.063926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a717ad7-dea4-4360-811a-b201933d7c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:14.064970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a717ad7-dea4-4360-811a-b201933d7c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:14.067078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a717ad7-dea4-4360-811a-b201933d7c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:14.067440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a717ad7-dea4-4360-811a-b201933d7c1c admin admin] PATH_INFO: `/v3/projects/343afb1e5ab3446eb16ba680491e692e/users/0b37f08b619f45229f55f27f7682a9c2/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:14.071500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a717ad7-dea4-4360-811a-b201933d7c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=343afb1e5ab3446eb16ba680491e692e, user_id=0b37f08b619f45229f55f27f7682a9c2, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:14.082290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a717ad7-dea4-4360-811a-b201933d7c1c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:14.113085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 822/1648] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:13 2022] PUT /identity/v3/projects/343afb1e5ab3446eb16ba680491e692e/users/0b37f08b619f45229f55f27f7682a9c2/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:15.706086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a46ad-8a1b-468f-9372-5710d2042668 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:15.707900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1a1a46ad-8a1b-468f-9372-5710d2042668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:15.722173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a46ad-8a1b-468f-9372-5710d2042668 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:15.813325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:15.814003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:15.814291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:15.822704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] PATH_INFO: `/v3/projects/415cd9ddca8f406d898b1badb80fdb90` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:15.824459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=415cd9ddca8f406d898b1badb80fdb90)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:15.825466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e0d5e-0b17-47e5-acca-317cdbf5e32e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:15.826171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 05:27:15.826171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 21 05:27:15.826171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: Sep 21 05:27:15.826442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e0d5e-0b17-47e5-acca-317cdbf5e32e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:15.826442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e0d5e-0b17-47e5-acca-317cdbf5e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:15.830719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:15.869126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-878e0d5e-0b17-47e5-acca-317cdbf5e32e None None] MFA Rules not processed for user `0b37f08b619f45229f55f27f7682a9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:15.874925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-878e0d5e-0b17-47e5-acca-317cdbf5e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:15.959634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '415cd9ddca8f406d898b1badb80fdb90', 'request_id': 'req-3e3b706d-73eb-45dd-963b-46da3b291e78'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:15.979443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '415cd9ddca8f406d898b1badb80fdb90', 'request_id': 'req-3e3b706d-73eb-45dd-963b-46da3b291e78'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:15.979950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:15.992122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '415cd9ddca8f406d898b1badb80fdb90', 'request_id': 'req-3e3b706d-73eb-45dd-963b-46da3b291e78'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:16.045675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] The token cache is being invalidate because project 415cd9ddca8f406d898b1badb80fdb90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:16.046904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:16.047714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3e3b706d-73eb-45dd-963b-46da3b291e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:16.051814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 827/1649] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:15 2022] DELETE /identity/v3/projects/415cd9ddca8f406d898b1badb80fdb90 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:16.060972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffa718-cc9d-493f-b209-6c383fdd99b3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:16.061268 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffa718-cc9d-493f-b209-6c383fdd99b3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:16.061536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffa718-cc9d-493f-b209-6c383fdd99b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:16.062310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 823/1650] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:15 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:16.086259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4bffa718-cc9d-493f-b209-6c383fdd99b3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:16.087647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4bffa718-cc9d-493f-b209-6c383fdd99b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:16.116118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:16.219748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 828/1651] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:16.221503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222dd64d-e517-4d04-8447-75adf2f6f37d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:16.221788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222dd64d-e517-4d04-8447-75adf2f6f37d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:16.222104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222dd64d-e517-4d04-8447-75adf2f6f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:16.269373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-222dd64d-e517-4d04-8447-75adf2f6f37d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:16.273009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-222dd64d-e517-4d04-8447-75adf2f6f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:16.286326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 824/1652] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:16.290642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffa718-cc9d-493f-b209-6c383fdd99b3 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:16.305449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffa718-cc9d-493f-b209-6c383fdd99b3 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:16.322803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae8e18-9e67-4250-bc67-ac9f917e8c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:16.328633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae8e18-9e67-4250-bc67-ac9f917e8c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:16.333009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae8e18-9e67-4250-bc67-ac9f917e8c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:16.333537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae8e18-9e67-4250-bc67-ac9f917e8c5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:16.334796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae8e18-9e67-4250-bc67-ac9f917e8c5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:16.337012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae8e18-9e67-4250-bc67-ac9f917e8c5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:16.340682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae8e18-9e67-4250-bc67-ac9f917e8c5c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:16.374101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222dd64d-e517-4d04-8447-75adf2f6f37d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:16.405961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 829/1653] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:27:16.412328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222dd64d-e517-4d04-8447-75adf2f6f37d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:16.436453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02156253-0209-4a05-89b1-36516b89f8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:16.443563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02156253-0209-4a05-89b1-36516b89f8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:16.444246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02156253-0209-4a05-89b1-36516b89f8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:16.444697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02156253-0209-4a05-89b1-36516b89f8c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:16.445809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02156253-0209-4a05-89b1-36516b89f8c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:16.449248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02156253-0209-4a05-89b1-36516b89f8c8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:16.503950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae8e18-9e67-4250-bc67-ac9f917e8c5c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:16.510510 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 825/1654] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:16 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:16.524706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae8e18-9e67-4250-bc67-ac9f917e8c5c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:16.692145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7b546-be75-451b-ab37-d98883b0a354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:16.692873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7b546-be75-451b-ab37-d98883b0a354 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:16.693320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7b546-be75-451b-ab37-d98883b0a354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:16.693799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7b546-be75-451b-ab37-d98883b0a354 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:16.694742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7b546-be75-451b-ab37-d98883b0a354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:16.696888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7b546-be75-451b-ab37-d98883b0a354 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:16.744435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 830/1655] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:16 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:16.745499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02156253-0209-4a05-89b1-36516b89f8c8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:16.769348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02156253-0209-4a05-89b1-36516b89f8c8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:16.788113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc658e7-c874-4e25-a4eb-311c6dcc02fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:16.788927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc658e7-c874-4e25-a4eb-311c6dcc02fb admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:16.789313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc658e7-c874-4e25-a4eb-311c6dcc02fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:16.789688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc658e7-c874-4e25-a4eb-311c6dcc02fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:16.790481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc658e7-c874-4e25-a4eb-311c6dcc02fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:16.792274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc658e7-c874-4e25-a4eb-311c6dcc02fb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:16.814140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 826/1656] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:27:16.817905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7b546-be75-451b-ab37-d98883b0a354 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:16.880914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7b546-be75-451b-ab37-d98883b0a354 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:16.923372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e947388-b551-45e5-ae14-e8f3630bba5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:16.926767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e947388-b551-45e5-ae14-e8f3630bba5e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:16.933286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e947388-b551-45e5-ae14-e8f3630bba5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:16.933432 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e947388-b551-45e5-ae14-e8f3630bba5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:16.938177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e947388-b551-45e5-ae14-e8f3630bba5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:16.947110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e947388-b551-45e5-ae14-e8f3630bba5e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:16.995348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 831/1657] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:16.996667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc658e7-c874-4e25-a4eb-311c6dcc02fb admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:17.020692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc658e7-c874-4e25-a4eb-311c6dcc02fb admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:17.044086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74101c0a-b798-4b9c-a299-210d741d91e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:17.045033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74101c0a-b798-4b9c-a299-210d741d91e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:17.045318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74101c0a-b798-4b9c-a299-210d741d91e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:17.045680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74101c0a-b798-4b9c-a299-210d741d91e8 admin admin] PATH_INFO: `/v3/projects/474d54a1827748d6a21422ace20a70c9/users/bcab9aa11b2b4fc69412fa749919f2bd/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:17.111168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74101c0a-b798-4b9c-a299-210d741d91e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474d54a1827748d6a21422ace20a70c9, user_id=bcab9aa11b2b4fc69412fa749919f2bd, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:17.120075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74101c0a-b798-4b9c-a299-210d741d91e8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:17.160334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 827/1658] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:16 2022] PUT /identity/v3/projects/474d54a1827748d6a21422ace20a70c9/users/bcab9aa11b2b4fc69412fa749919f2bd/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:17.161307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8bebb-0681-458c-8b50-9a960fbf62d5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:17.162457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8bebb-0681-458c-8b50-9a960fbf62d5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:17.162457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8bebb-0681-458c-8b50-9a960fbf62d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:17.233326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d4e8bebb-0681-458c-8b50-9a960fbf62d5 None None] MFA Rules not processed for user `bcab9aa11b2b4fc69412fa749919f2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:17.234904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d4e8bebb-0681-458c-8b50-9a960fbf62d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:17.422872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 832/1659] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:17.427743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74101c0a-b798-4b9c-a299-210d741d91e8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:17.663504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74101c0a-b798-4b9c-a299-210d741d91e8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:17.669795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6c0e3-009d-4d0b-be48-bd815e53f1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:17.670611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6c0e3-009d-4d0b-be48-bd815e53f1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:17.670906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6c0e3-009d-4d0b-be48-bd815e53f1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:17.671180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6c0e3-009d-4d0b-be48-bd815e53f1e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:17.671965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6c0e3-009d-4d0b-be48-bd815e53f1e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:17.675206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6c0e3-009d-4d0b-be48-bd815e53f1e4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:17.796039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 828/1660] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:17 2022] POST /identity/v3/projects => generated 371 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:17.799834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8bebb-0681-458c-8b50-9a960fbf62d5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:17.836982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8bebb-0681-458c-8b50-9a960fbf62d5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:17.946748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:18.147380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4279c7-3394-4583-89d6-5b0751d05d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:18.148283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4279c7-3394-4583-89d6-5b0751d05d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:18.148662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4279c7-3394-4583-89d6-5b0751d05d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:18.152106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4279c7-3394-4583-89d6-5b0751d05d2d admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:18.152806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4279c7-3394-4583-89d6-5b0751d05d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:18.156806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4279c7-3394-4583-89d6-5b0751d05d2d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:18.216734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 833/1661] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:17 2022] POST /identity/v3/users => generated 292 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:18.221855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6c0e3-009d-4d0b-be48-bd815e53f1e4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:18.239083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6c0e3-009d-4d0b-be48-bd815e53f1e4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:18.255127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caecda4-9a9d-4062-ac84-ba104e61a02e None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:18.255415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caecda4-9a9d-4062-ac84-ba104e61a02e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:18.255693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caecda4-9a9d-4062-ac84-ba104e61a02e None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:18.257566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 834/1662] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:27:18.268241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d338fb-17c4-4562-bbf7-2629e5042899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:18.269168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d338fb-17c4-4562-bbf7-2629e5042899 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:18.269584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d338fb-17c4-4562-bbf7-2629e5042899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:18.270035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d338fb-17c4-4562-bbf7-2629e5042899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:18.270967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d338fb-17c4-4562-bbf7-2629e5042899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:18.273082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d338fb-17c4-4562-bbf7-2629e5042899 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:18.308055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caecda4-9a9d-4062-ac84-ba104e61a02e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:18.310512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 829/1663] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:18.316598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d338fb-17c4-4562-bbf7-2629e5042899 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:18.326940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e5d338fb-17c4-4562-bbf7-2629e5042899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:18.329345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caecda4-9a9d-4062-ac84-ba104e61a02e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:18.349809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5d690-7ad9-43d2-aab1-ebd3154c753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:18.350602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5d690-7ad9-43d2-aab1-ebd3154c753d admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:18.350953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5d690-7ad9-43d2-aab1-ebd3154c753d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:18.351300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5d690-7ad9-43d2-aab1-ebd3154c753d admin admin] PATH_INFO: `/v3/projects/100c3670ae954300bded6c315d9c9b18/users/f47bd7c7f4854eb28623af74a386c2a2/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:18.367553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d338fb-17c4-4562-bbf7-2629e5042899 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:18.379766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:18.380762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:18.381181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:18.381604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] PATH_INFO: `/v3/users/c8ccedc126aa487f8fdfb50ddfd1b264` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:18.383398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ccedc126aa487f8fdfb50ddfd1b264)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:18.386253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:18.392034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5d690-7ad9-43d2-aab1-ebd3154c753d admin admin] RBAC: Authorizing `identity:create_grant(project_id=100c3670ae954300bded6c315d9c9b18, user_id=f47bd7c7f4854eb28623af74a386c2a2, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:18.396059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5d690-7ad9-43d2-aab1-ebd3154c753d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:18.431219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 835/1664] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:18 2022] PUT /identity/v3/projects/100c3670ae954300bded6c315d9c9b18/users/f47bd7c7f4854eb28623af74a386c2a2/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:18.434290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2becba1-81ed-44a4-96cc-03624e77f4bb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:18.434618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2becba1-81ed-44a4-96cc-03624e77f4bb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:18.435248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2becba1-81ed-44a4-96cc-03624e77f4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:18.467892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e2becba1-81ed-44a4-96cc-03624e77f4bb None None] MFA Rules not processed for user `4b3d8a1c8b6640ae9e90ece0bbfbf410`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:18.471410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e2becba1-81ed-44a4-96cc-03624e77f4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:18.496679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:18.503894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ccedc126aa487f8fdfb50ddfd1b264'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:18.504999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ccedc126aa487f8fdfb50ddfd1b264'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:18.520486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ccedc126aa487f8fdfb50ddfd1b264'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:18.521918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ccedc126aa487f8fdfb50ddfd1b264'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:18.534092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ccedc126aa487f8fdfb50ddfd1b264'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:18.581219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 830/1665] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:18 2022] DELETE /identity/v3/users/c8ccedc126aa487f8fdfb50ddfd1b264 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:18.582438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:18.594189 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 836/1666] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:27:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:18.596136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ae393-9b65-4214-9fea-e8decd0f64c8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:18.596533 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ae393-9b65-4214-9fea-e8decd0f64c8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:18.596913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ae393-9b65-4214-9fea-e8decd0f64c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:18.622738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:18.624045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8b8ae393-9b65-4214-9fea-e8decd0f64c8 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:18.629539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8b8ae393-9b65-4214-9fea-e8decd0f64c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:18.778651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e387a-578c-4e1e-aaf6-6997ece67092 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:18.783305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 837/1667] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:18.784511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ae393-9b65-4214-9fea-e8decd0f64c8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:18.785482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43421d6-50ba-4983-abe3-558f8e29ce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:18.786407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43421d6-50ba-4983-abe3-558f8e29ce26 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:18.786980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43421d6-50ba-4983-abe3-558f8e29ce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:18.788486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43421d6-50ba-4983-abe3-558f8e29ce26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:18.792171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43421d6-50ba-4983-abe3-558f8e29ce26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:18.795308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8b8ae393-9b65-4214-9fea-e8decd0f64c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:18.796415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43421d6-50ba-4983-abe3-558f8e29ce26 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:18.821770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 831/1668] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:18.824619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43421d6-50ba-4983-abe3-558f8e29ce26 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:18.839249 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43421d6-50ba-4983-abe3-558f8e29ce26 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:18.846271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf227de2-ba90-4067-9521-76b962eddb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:18.849761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf227de2-ba90-4067-9521-76b962eddb32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:18.851618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:18.859127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf227de2-ba90-4067-9521-76b962eddb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:18.864436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf227de2-ba90-4067-9521-76b962eddb32 admin admin] PATH_INFO: `/v3/projects/100c3670ae954300bded6c315d9c9b18/users/f47bd7c7f4854eb28623af74a386c2a2/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:18.871177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf227de2-ba90-4067-9521-76b962eddb32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=100c3670ae954300bded6c315d9c9b18, user_id=f47bd7c7f4854eb28623af74a386c2a2, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:18.875022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf227de2-ba90-4067-9521-76b962eddb32 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:18.902268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 832/1669] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:18 2022] PUT /identity/v3/projects/100c3670ae954300bded6c315d9c9b18/users/f47bd7c7f4854eb28623af74a386c2a2/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:18.903482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf227de2-ba90-4067-9521-76b962eddb32 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:18.909357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bf227de2-ba90-4067-9521-76b962eddb32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:18.954719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ae393-9b65-4214-9fea-e8decd0f64c8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:19.102189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf227de2-ba90-4067-9521-76b962eddb32 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:19.107366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43824c-f9c6-4294-b459-a7651c74cccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:19.108482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43824c-f9c6-4294-b459-a7651c74cccd service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:19.108907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43824c-f9c6-4294-b459-a7651c74cccd service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:19.109324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43824c-f9c6-4294-b459-a7651c74cccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:19.155023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ec062-68e0-4b86-9a9a-0341801ed148 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:19.155672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ec062-68e0-4b86-9a9a-0341801ed148 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:19.155959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ec062-68e0-4b86-9a9a-0341801ed148 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:19.156238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ec062-68e0-4b86-9a9a-0341801ed148 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:19.192077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ec062-68e0-4b86-9a9a-0341801ed148 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:19.194166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ec062-68e0-4b86-9a9a-0341801ed148 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:19.250551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 838/1670] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:19.256235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2740a5-08ee-44b3-9a12-6c183dfd843e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:19.256567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2740a5-08ee-44b3-9a12-6c183dfd843e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:19.271667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2740a5-08ee-44b3-9a12-6c183dfd843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:19.398082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:19.398964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fd2740a5-08ee-44b3-9a12-6c183dfd843e None None] MFA Rules not processed for user `f47bd7c7f4854eb28623af74a386c2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:19.400078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fd2740a5-08ee-44b3-9a12-6c183dfd843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:19.417754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43824c-f9c6-4294-b459-a7651c74cccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:19.423191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43824c-f9c6-4294-b459-a7651c74cccd service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:19.487021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 833/1671] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:19.711998 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43824c-f9c6-4294-b459-a7651c74cccd service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:19.715059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7c43824c-f9c6-4294-b459-a7651c74cccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:19.732880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 839/1672] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:19.844116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43824c-f9c6-4294-b459-a7651c74cccd service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:19.848870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68619f20-9c1f-4fbd-b5bb-2e35192b6326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:19.849646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68619f20-9c1f-4fbd-b5bb-2e35192b6326 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:19.849952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68619f20-9c1f-4fbd-b5bb-2e35192b6326 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:19.850235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68619f20-9c1f-4fbd-b5bb-2e35192b6326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:19.864881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68619f20-9c1f-4fbd-b5bb-2e35192b6326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:19.866923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68619f20-9c1f-4fbd-b5bb-2e35192b6326 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:19.907516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 834/1673] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:20.010110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2740a5-08ee-44b3-9a12-6c183dfd843e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:20.011765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fd2740a5-08ee-44b3-9a12-6c183dfd843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:20.039886 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2740a5-08ee-44b3-9a12-6c183dfd843e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:20.058340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:20.058897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:20.059179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:20.059458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] PATH_INFO: `/v3/users/035b773f4fec49babaf6418272cfde73` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:20.061104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=035b773f4fec49babaf6418272cfde73)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:20.063284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:20.068976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68619f20-9c1f-4fbd-b5bb-2e35192b6326 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:20.091949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68619f20-9c1f-4fbd-b5bb-2e35192b6326 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:20.105890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:20.106793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:20.107312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:20.108359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:20.113736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:20.143104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:20.164472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '035b773f4fec49babaf6418272cfde73'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:20.186537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '035b773f4fec49babaf6418272cfde73'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:20.187227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '035b773f4fec49babaf6418272cfde73'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:20.187908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '035b773f4fec49babaf6418272cfde73'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:20.203826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '035b773f4fec49babaf6418272cfde73'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:20.240262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 840/1674] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:20 2022] DELETE /identity/v3/users/035b773f4fec49babaf6418272cfde73 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:20.271484 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:20.283274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:20.305405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 835/1675] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:20.626821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:20.630006 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:20.703899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:20.705351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:20.754347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e9486-b084-41fa-826c-b584d3b19f3c service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:20.892300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ac64b-148a-4dae-9fa4-e4e09081b488 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:20.912871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:20.915251 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:20.915600 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:20.923032 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] PATH_INFO: `/v3/projects/50c624b022e8481f8ce19587e1149b54` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:20.932310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c624b022e8481f8ce19587e1149b54)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:20.936358 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:20.996322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:20.996991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:20.997267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:20.997653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] PATH_INFO: `/v3/projects/91291204c34f46449d96ca93d93672e4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:21.004652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] RBAC: Authorizing `identity:delete_project(project_id=91291204c34f46449d96ca93d93672e4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:21.006791 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:21.090140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c624b022e8481f8ce19587e1149b54', 'request_id': 'req-9046728c-d223-43e6-b698-2634dcd04924'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.111848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c624b022e8481f8ce19587e1149b54', 'request_id': 'req-9046728c-d223-43e6-b698-2634dcd04924'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.133894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c624b022e8481f8ce19587e1149b54', 'request_id': 'req-9046728c-d223-43e6-b698-2634dcd04924'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.138787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:21.180596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] The token cache is being invalidate because project 50c624b022e8481f8ce19587e1149b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:21.180889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.182578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91291204c34f46449d96ca93d93672e4', 'request_id': 'req-cf1ec6ac-97b6-4911-9681-f12ec141feba'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.184659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.186391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 841/1676] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:20 2022] DELETE /identity/v3/projects/50c624b022e8481f8ce19587e1149b54 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:21.192813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:21.198996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:21.203321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91291204c34f46449d96ca93d93672e4', 'request_id': 'req-cf1ec6ac-97b6-4911-9681-f12ec141feba'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.220932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91291204c34f46449d96ca93d93672e4', 'request_id': 'req-cf1ec6ac-97b6-4911-9681-f12ec141feba'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.248675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] The token cache is being invalidate because project 91291204c34f46449d96ca93d93672e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:21.248965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.251249 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.257702 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 836/1677] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:20 2022] DELETE /identity/v3/projects/91291204c34f46449d96ca93d93672e4 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:27:21.257702 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:21.259539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:21.329644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:21.362373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046728c-d223-43e6-b698-2634dcd04924 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:21.415963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ec6ac-97b6-4911-9681-f12ec141feba admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:21.424070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:21.424967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:21.426029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:21.426518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] PATH_INFO: `/v3/users/0b37f08b619f45229f55f27f7682a9c2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:21.428047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b37f08b619f45229f55f27f7682a9c2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:21.431089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:21.448500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:21.449350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:21.449729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:21.451321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] PATH_INFO: `/v3/users/d3ff88c27ad24678b61bc04281a8d30f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:21.452769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ff88c27ad24678b61bc04281a8d30f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:21.455560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:21.523740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b37f08b619f45229f55f27f7682a9c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.524444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b37f08b619f45229f55f27f7682a9c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.537275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b37f08b619f45229f55f27f7682a9c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.538393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b37f08b619f45229f55f27f7682a9c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.549490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ff88c27ad24678b61bc04281a8d30f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.551612 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9d3522c-db1e-41c2-8684-b9ab7a38b8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b37f08b619f45229f55f27f7682a9c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.575590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ff88c27ad24678b61bc04281a8d30f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.576522 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ff88c27ad24678b61bc04281a8d30f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.579748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ff88c27ad24678b61bc04281a8d30f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.601095 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-cf4e83b8-d082-4327-8561-1d39306a56e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ff88c27ad24678b61bc04281a8d30f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:21.606106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 837/1678] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:21 2022] DELETE /identity/v3/users/0b37f08b619f45229f55f27f7682a9c2 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:21.609740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686503c8-355b-404e-9bf3-d9222d12df07 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:21.610448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686503c8-355b-404e-9bf3-d9222d12df07 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:21.610986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686503c8-355b-404e-9bf3-d9222d12df07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:21.646596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-686503c8-355b-404e-9bf3-d9222d12df07 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:21.648599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-686503c8-355b-404e-9bf3-d9222d12df07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:21.648844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 842/1679] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:21 2022] DELETE /identity/v3/users/d3ff88c27ad24678b61bc04281a8d30f => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:21.654486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071efbd-4d82-49ac-8c55-fa2df127acae None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:21.654927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071efbd-4d82-49ac-8c55-fa2df127acae None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:21.655303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071efbd-4d82-49ac-8c55-fa2df127acae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:21.684371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3071efbd-4d82-49ac-8c55-fa2df127acae None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:21.685759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3071efbd-4d82-49ac-8c55-fa2df127acae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:21.734568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:21.773966 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 838/1680] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:21.782019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d7ebc-f881-49e3-a438-ec8e510ee12d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:21.782339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d7ebc-f881-49e3-a438-ec8e510ee12d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:21.782619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d7ebc-f881-49e3-a438-ec8e510ee12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:21.811687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 843/1681] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:21.829293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c70d7ebc-f881-49e3-a438-ec8e510ee12d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:21.830735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c70d7ebc-f881-49e3-a438-ec8e510ee12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:21.840806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071efbd-4d82-49ac-8c55-fa2df127acae None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:21.842357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3071efbd-4d82-49ac-8c55-fa2df127acae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:21.868158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:21.876601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 839/1682] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:21.878561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d7ebc-f881-49e3-a438-ec8e510ee12d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:21.894147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d7ebc-f881-49e3-a438-ec8e510ee12d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:21.904050 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0a642-5a33-4a9d-a64a-88ffbad986e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:21.904684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0a642-5a33-4a9d-a64a-88ffbad986e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:21.904960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0a642-5a33-4a9d-a64a-88ffbad986e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:21.905297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0a642-5a33-4a9d-a64a-88ffbad986e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:21.906189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0a642-5a33-4a9d-a64a-88ffbad986e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:21.906455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0a642-5a33-4a9d-a64a-88ffbad986e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:21.908789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0a642-5a33-4a9d-a64a-88ffbad986e9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:21.930215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 840/1683] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:21.933143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0a642-5a33-4a9d-a64a-88ffbad986e9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:21.952192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0a642-5a33-4a9d-a64a-88ffbad986e9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:21.964674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8bdb2-ff0c-4ce5-8842-6dffc1d68cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:21.965308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8bdb2-ff0c-4ce5-8842-6dffc1d68cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:21.965591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8bdb2-ff0c-4ce5-8842-6dffc1d68cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:21.966080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8bdb2-ff0c-4ce5-8842-6dffc1d68cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:21.966740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8bdb2-ff0c-4ce5-8842-6dffc1d68cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:21.968714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8bdb2-ff0c-4ce5-8842-6dffc1d68cbb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:22.006328 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071efbd-4d82-49ac-8c55-fa2df127acae None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:22.015460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74570ab-c656-406c-b538-1ad6addc5742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:22.016433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74570ab-c656-406c-b538-1ad6addc5742 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:22.017982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74570ab-c656-406c-b538-1ad6addc5742 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:22.019091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74570ab-c656-406c-b538-1ad6addc5742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:22.046500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 841/1684] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:21 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:27:22.046865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74570ab-c656-406c-b538-1ad6addc5742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:22.049100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8bdb2-ff0c-4ce5-8842-6dffc1d68cbb admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:22.050954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74570ab-c656-406c-b538-1ad6addc5742 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:22.074612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 844/1685] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:27:22.147136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8bdb2-ff0c-4ce5-8842-6dffc1d68cbb admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:22.155596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df125c-53b7-4727-9aad-1431f34f4748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:22.156463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df125c-53b7-4727-9aad-1431f34f4748 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:22.156901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df125c-53b7-4727-9aad-1431f34f4748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:22.157276 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df125c-53b7-4727-9aad-1431f34f4748 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:22.158239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df125c-53b7-4727-9aad-1431f34f4748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:22.161339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df125c-53b7-4727-9aad-1431f34f4748 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:22.228373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 842/1686] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:22 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:22.231200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74570ab-c656-406c-b538-1ad6addc5742 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:22.257212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74570ab-c656-406c-b538-1ad6addc5742 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:22.262159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a96e5-f0d1-43fb-b8ab-733f1e74d1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:22.262933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a96e5-f0d1-43fb-b8ab-733f1e74d1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:22.264086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a96e5-f0d1-43fb-b8ab-733f1e74d1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:22.264498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a96e5-f0d1-43fb-b8ab-733f1e74d1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:22.267275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a96e5-f0d1-43fb-b8ab-733f1e74d1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:22.269431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a96e5-f0d1-43fb-b8ab-733f1e74d1d4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:22.295458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 845/1687] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:22.296717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df125c-53b7-4727-9aad-1431f34f4748 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:22.310701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df125c-53b7-4727-9aad-1431f34f4748 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:22.321527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e55186-eb1c-4ea7-ace7-9ab1979c5f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:22.323650 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e55186-eb1c-4ea7-ace7-9ab1979c5f4e admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:22.324629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e55186-eb1c-4ea7-ace7-9ab1979c5f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:22.325002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e55186-eb1c-4ea7-ace7-9ab1979c5f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:22.326008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e55186-eb1c-4ea7-ace7-9ab1979c5f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:22.351452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e55186-eb1c-4ea7-ace7-9ab1979c5f4e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:22.374979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 843/1688] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:22.376075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a96e5-f0d1-43fb-b8ab-733f1e74d1d4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:22.401043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a96e5-f0d1-43fb-b8ab-733f1e74d1d4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:22.407455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b294438-d9af-4a34-ab88-d362c077c1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:22.408485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294438-d9af-4a34-ab88-d362c077c1fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:22.408997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294438-d9af-4a34-ab88-d362c077c1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:22.409476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294438-d9af-4a34-ab88-d362c077c1fd admin admin] PATH_INFO: `/v3/projects/089de397f5ff4e26b191403a0341f65e/users/609d915f4d014d51aa1d8182df46ddaf/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:22.458910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b294438-d9af-4a34-ab88-d362c077c1fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=089de397f5ff4e26b191403a0341f65e, user_id=609d915f4d014d51aa1d8182df46ddaf, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:22.461785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b294438-d9af-4a34-ab88-d362c077c1fd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:22.498426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 846/1689] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:22 2022] PUT /identity/v3/projects/089de397f5ff4e26b191403a0341f65e/users/609d915f4d014d51aa1d8182df46ddaf/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:22.512610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262897d-1bd4-4f1e-b74e-484ac8ca6f57 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:22.513015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262897d-1bd4-4f1e-b74e-484ac8ca6f57 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:22.513369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262897d-1bd4-4f1e-b74e-484ac8ca6f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:22.579496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0262897d-1bd4-4f1e-b74e-484ac8ca6f57 None None] MFA Rules not processed for user `609d915f4d014d51aa1d8182df46ddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:22.581137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0262897d-1bd4-4f1e-b74e-484ac8ca6f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:22.851574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 844/1690] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:22.854734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b294438-d9af-4a34-ab88-d362c077c1fd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:23.005821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b294438-d9af-4a34-ab88-d362c077c1fd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:23.017299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7de00e-b594-4f0b-b043-51731877d744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:23.018080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7de00e-b594-4f0b-b043-51731877d744 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:23.018627 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7de00e-b594-4f0b-b043-51731877d744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:23.019133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7de00e-b594-4f0b-b043-51731877d744 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:23.020010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7de00e-b594-4f0b-b043-51731877d744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:23.024992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7de00e-b594-4f0b-b043-51731877d744 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:23.056991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262897d-1bd4-4f1e-b74e-484ac8ca6f57 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:23.059001 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0262897d-1bd4-4f1e-b74e-484ac8ca6f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:23.087647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262897d-1bd4-4f1e-b74e-484ac8ca6f57 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:23.109159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 847/1691] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:22 2022] POST /identity/v3/projects => generated 371 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:23.110183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:23.111078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:23.111817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:23.112293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] PATH_INFO: `/v3/projects/343afb1e5ab3446eb16ba680491e692e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:23.113917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7de00e-b594-4f0b-b043-51731877d744 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:23.115188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] RBAC: Authorizing `identity:delete_project(project_id=343afb1e5ab3446eb16ba680491e692e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:23.119591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:23.136233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:23.231789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '343afb1e5ab3446eb16ba680491e692e', 'request_id': 'req-8fe9021b-d0b8-4837-9fea-3b9b7043426e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:23.248168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '343afb1e5ab3446eb16ba680491e692e', 'request_id': 'req-8fe9021b-d0b8-4837-9fea-3b9b7043426e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:23.259109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '343afb1e5ab3446eb16ba680491e692e', 'request_id': 'req-8fe9021b-d0b8-4837-9fea-3b9b7043426e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:23.285188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] The token cache is being invalidate because project 343afb1e5ab3446eb16ba680491e692e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:23.285684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:23.286518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8fe9021b-d0b8-4837-9fea-3b9b7043426e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:23.298568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 845/1692] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:23 2022] DELETE /identity/v3/projects/343afb1e5ab3446eb16ba680491e692e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:23.300583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366ec16-208e-44a8-b7a8-7716de1397fa None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:23.300981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366ec16-208e-44a8-b7a8-7716de1397fa None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:23.301368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366ec16-208e-44a8-b7a8-7716de1397fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:23.336034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7de00e-b594-4f0b-b043-51731877d744 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:23.343848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d366ec16-208e-44a8-b7a8-7716de1397fa None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:23.345554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d366ec16-208e-44a8-b7a8-7716de1397fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:23.357130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:23.518537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33c150-41af-4b27-ae21-1480a47777f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:23.519459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33c150-41af-4b27-ae21-1480a47777f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:23.519459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33c150-41af-4b27-ae21-1480a47777f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:23.519727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33c150-41af-4b27-ae21-1480a47777f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:23.520383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33c150-41af-4b27-ae21-1480a47777f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:23.525335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33c150-41af-4b27-ae21-1480a47777f9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:23.561331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 846/1693] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:23.562217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f12cdd-f40d-4b4d-8d40-8aa22f58162a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:23.562833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f12cdd-f40d-4b4d-8d40-8aa22f58162a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:23.565743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f12cdd-f40d-4b4d-8d40-8aa22f58162a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:23.667114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 848/1694] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:23 2022] POST /identity/v3/users => generated 292 bytes in 551 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:23.674168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33c150-41af-4b27-ae21-1480a47777f9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:23.683014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0b33c150-41af-4b27-ae21-1480a47777f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:23.689577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c5f12cdd-f40d-4b4d-8d40-8aa22f58162a None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:23.700203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c5f12cdd-f40d-4b4d-8d40-8aa22f58162a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:23.730104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 847/1695] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:23.735889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f12cdd-f40d-4b4d-8d40-8aa22f58162a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:23.738281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33c150-41af-4b27-ae21-1480a47777f9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:23.747124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c5f12cdd-f40d-4b4d-8d40-8aa22f58162a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:23.763134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:23.763801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:23.764088 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:23.764362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] PATH_INFO: `/v3/projects/28f31a6be8f8416f8f593eb7a0474d86` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:23.771661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f31a6be8f8416f8f593eb7a0474d86)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:23.773829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:23.792358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f12cdd-f40d-4b4d-8d40-8aa22f58162a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:23.812200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f0648-007f-41a2-9184-71512c314e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:23.812837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0648-007f-41a2-9184-71512c314e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:23.813113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0648-007f-41a2-9184-71512c314e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:23.813383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0648-007f-41a2-9184-71512c314e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:23.827138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f0648-007f-41a2-9184-71512c314e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:23.838536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f0648-007f-41a2-9184-71512c314e16 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:23.894753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 848/1696] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:23.901198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f0648-007f-41a2-9184-71512c314e16 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:23.943181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f0648-007f-41a2-9184-71512c314e16 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:23.979357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f31a6be8f8416f8f593eb7a0474d86', 'request_id': 'req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:23.998228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f31a6be8f8416f8f593eb7a0474d86', 'request_id': 'req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:24.006396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f31a6be8f8416f8f593eb7a0474d86', 'request_id': 'req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:24.034682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] The token cache is being invalidate because project 28f31a6be8f8416f8f593eb7a0474d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:24.035995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:24.038276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:24.040465 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 849/1697] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:23 2022] DELETE /identity/v3/projects/28f31a6be8f8416f8f593eb7a0474d86 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:24.042380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:24.047154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:24.098576 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8925665-7085-48bf-8999-404833c24984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:24.099208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8925665-7085-48bf-8999-404833c24984 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.099502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8925665-7085-48bf-8999-404833c24984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.099767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8925665-7085-48bf-8999-404833c24984 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.100497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8925665-7085-48bf-8999-404833c24984 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:24.100805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8925665-7085-48bf-8999-404833c24984 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:24.113401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8925665-7085-48bf-8999-404833c24984 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:24.144433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 849/1698] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:24.147375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925136d8-4ba6-49e3-a9eb-00df3d32da18 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.148266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925136d8-4ba6-49e3-a9eb-00df3d32da18 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.150095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925136d8-4ba6-49e3-a9eb-00df3d32da18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.217674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-925136d8-4ba6-49e3-a9eb-00df3d32da18 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:24.223562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-925136d8-4ba6-49e3-a9eb-00df3d32da18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:24.252787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb13dc7-30b8-4b08-a57c-29db6abf9ded admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:24.263296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 850/1699] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:24.264158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925136d8-4ba6-49e3-a9eb-00df3d32da18 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:24.269443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-925136d8-4ba6-49e3-a9eb-00df3d32da18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:24.272258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ae06a-0e5d-4f8d-97e1-27e4eb1d019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:24.274652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ae06a-0e5d-4f8d-97e1-27e4eb1d019d admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.276526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ae06a-0e5d-4f8d-97e1-27e4eb1d019d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.276968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ae06a-0e5d-4f8d-97e1-27e4eb1d019d admin admin] PATH_INFO: `/v3/projects/2afd0f4e012848349b699e0a0ebc2956/users/9c803c822bb04ab89be9440e4f6ba99f/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.283489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:24.297968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925136d8-4ba6-49e3-a9eb-00df3d32da18 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:24.307081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382197ea-1ffc-483b-9d44-2021bef64bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:24.307711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382197ea-1ffc-483b-9d44-2021bef64bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.308002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382197ea-1ffc-483b-9d44-2021bef64bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.309297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382197ea-1ffc-483b-9d44-2021bef64bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.309969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382197ea-1ffc-483b-9d44-2021bef64bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:24.312536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382197ea-1ffc-483b-9d44-2021bef64bbe admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:24.317621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ae06a-0e5d-4f8d-97e1-27e4eb1d019d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2afd0f4e012848349b699e0a0ebc2956, user_id=9c803c822bb04ab89be9440e4f6ba99f, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:24.322489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ae06a-0e5d-4f8d-97e1-27e4eb1d019d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:24.341860 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 850/1700] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:24 2022] PUT /identity/v3/projects/2afd0f4e012848349b699e0a0ebc2956/users/9c803c822bb04ab89be9440e4f6ba99f/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:24.343403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e71a0a-05f1-45ed-916b-1ea337117e6d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.343746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e71a0a-05f1-45ed-916b-1ea337117e6d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.344211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e71a0a-05f1-45ed-916b-1ea337117e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.361701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:24.371559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e0e71a0a-05f1-45ed-916b-1ea337117e6d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:24.372130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 851/1701] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:24 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:24.372995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382197ea-1ffc-483b-9d44-2021bef64bbe admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:24.383457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e0e71a0a-05f1-45ed-916b-1ea337117e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:24.557295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382197ea-1ffc-483b-9d44-2021bef64bbe admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:24.567921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 851/1702] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:24.569219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e71a0a-05f1-45ed-916b-1ea337117e6d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:24.577297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528669d-6a1b-4543-957c-14d71555a3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:24.578107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528669d-6a1b-4543-957c-14d71555a3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.578503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528669d-6a1b-4543-957c-14d71555a3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.578916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528669d-6a1b-4543-957c-14d71555a3a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.580649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528669d-6a1b-4543-957c-14d71555a3a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:24.581765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528669d-6a1b-4543-957c-14d71555a3a6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:24.590185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e71a0a-05f1-45ed-916b-1ea337117e6d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:24.605411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 852/1703] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:24.610909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528669d-6a1b-4543-957c-14d71555a3a6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:24.612905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914ddd-dc17-4c6c-a3e1-045877701c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:24.613654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914ddd-dc17-4c6c-a3e1-045877701c0e admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.614079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914ddd-dc17-4c6c-a3e1-045877701c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.614462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914ddd-dc17-4c6c-a3e1-045877701c0e admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.615220 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56914ddd-dc17-4c6c-a3e1-045877701c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:24.622341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56914ddd-dc17-4c6c-a3e1-045877701c0e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:24.638178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528669d-6a1b-4543-957c-14d71555a3a6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:24.648371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d93bc-4506-4578-adde-ac1c0b4ea934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:24.649298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d93bc-4506-4578-adde-ac1c0b4ea934 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.649688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d93bc-4506-4578-adde-ac1c0b4ea934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.650144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d93bc-4506-4578-adde-ac1c0b4ea934 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.651099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d93bc-4506-4578-adde-ac1c0b4ea934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:24.651548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d93bc-4506-4578-adde-ac1c0b4ea934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:24.653720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d93bc-4506-4578-adde-ac1c0b4ea934 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:24.664572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 852/1704] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:24 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:24.665553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914ddd-dc17-4c6c-a3e1-045877701c0e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:24.687969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 853/1705] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:24.692200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914ddd-dc17-4c6c-a3e1-045877701c0e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:24.692920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d93bc-4506-4578-adde-ac1c0b4ea934 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:24.704646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85326e0-db33-4f7b-9e9f-1c662990d539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:24.705480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85326e0-db33-4f7b-9e9f-1c662990d539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.705876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85326e0-db33-4f7b-9e9f-1c662990d539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.707643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85326e0-db33-4f7b-9e9f-1c662990d539 admin admin] PATH_INFO: `/v3/projects/2afd0f4e012848349b699e0a0ebc2956/users/9c803c822bb04ab89be9440e4f6ba99f/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.711302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d93bc-4506-4578-adde-ac1c0b4ea934 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:24.713867 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85326e0-db33-4f7b-9e9f-1c662990d539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2afd0f4e012848349b699e0a0ebc2956, user_id=9c803c822bb04ab89be9440e4f6ba99f, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:24.716798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85326e0-db33-4f7b-9e9f-1c662990d539 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:24.725244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00f0ac-a304-4808-a68e-053ace786b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:24.726756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00f0ac-a304-4808-a68e-053ace786b2d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.728346 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00f0ac-a304-4808-a68e-053ace786b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.729911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00f0ac-a304-4808-a68e-053ace786b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.731041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e00f0ac-a304-4808-a68e-053ace786b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:24.740247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e00f0ac-a304-4808-a68e-053ace786b2d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:24.753714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 853/1706] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:24 2022] PUT /identity/v3/projects/2afd0f4e012848349b699e0a0ebc2956/users/9c803c822bb04ab89be9440e4f6ba99f/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:24.764128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85326e0-db33-4f7b-9e9f-1c662990d539 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:24.785058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 854/1707] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:24.788835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4090-7295-4b17-b19e-f92732611858 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.792356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4090-7295-4b17-b19e-f92732611858 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.792766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4090-7295-4b17-b19e-f92732611858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.870555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c7bd4090-7295-4b17-b19e-f92732611858 None None] MFA Rules not processed for user `9c803c822bb04ab89be9440e4f6ba99f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:24.871931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c7bd4090-7295-4b17-b19e-f92732611858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:24.933109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85326e0-db33-4f7b-9e9f-1c662990d539 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:24.946999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680e7de-2f12-4e35-ab1f-fb2106c5c085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:24.947608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680e7de-2f12-4e35-ab1f-fb2106c5c085 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:24.947890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680e7de-2f12-4e35-ab1f-fb2106c5c085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:24.948380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680e7de-2f12-4e35-ab1f-fb2106c5c085 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:24.949041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680e7de-2f12-4e35-ab1f-fb2106c5c085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:24.952015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680e7de-2f12-4e35-ab1f-fb2106c5c085 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:25.027012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 854/1708] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:24 2022] POST /identity/v3/projects => generated 379 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:25.028770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680e7de-2f12-4e35-ab1f-fb2106c5c085 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:25.058141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 855/1709] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:25.066290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd4090-7295-4b17-b19e-f92732611858 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:25.098831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd4090-7295-4b17-b19e-f92732611858 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:25.191974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:25.217453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680e7de-2f12-4e35-ab1f-fb2106c5c085 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:25.234196 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb882343-5c7d-4720-ad31-9dfe6e928eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:25.234728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb882343-5c7d-4720-ad31-9dfe6e928eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:25.235006 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb882343-5c7d-4720-ad31-9dfe6e928eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:25.235111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfa0d5-6d13-45c6-8666-85cd6d02c55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:25.235284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb882343-5c7d-4720-ad31-9dfe6e928eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:25.235829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfa0d5-6d13-45c6-8666-85cd6d02c55a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:25.235936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb882343-5c7d-4720-ad31-9dfe6e928eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:25.236113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfa0d5-6d13-45c6-8666-85cd6d02c55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:25.236389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfa0d5-6d13-45c6-8666-85cd6d02c55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:25.237292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfa0d5-6d13-45c6-8666-85cd6d02c55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:25.238109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb882343-5c7d-4720-ad31-9dfe6e928eb3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:25.238916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfa0d5-6d13-45c6-8666-85cd6d02c55a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:25.246638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:25.287483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 856/1710] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:25 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:25.289901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 855/1711] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:25.290204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb882343-5c7d-4720-ad31-9dfe6e928eb3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:25.292514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfa0d5-6d13-45c6-8666-85cd6d02c55a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:25.297221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-eb882343-5c7d-4720-ad31-9dfe6e928eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:25.320946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfa0d5-6d13-45c6-8666-85cd6d02c55a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:25.331207 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e198fb9-75a9-45c3-8c85-6696739be136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:25.332103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e198fb9-75a9-45c3-8c85-6696739be136 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:25.332491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e198fb9-75a9-45c3-8c85-6696739be136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:25.332877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e198fb9-75a9-45c3-8c85-6696739be136 admin admin] PATH_INFO: `/v3/projects/22ab9fea8cd34265a20d859ac1bdc026/users/7b355d17e49a4a04b2595ec3b0bbd7fa/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:25.371014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e198fb9-75a9-45c3-8c85-6696739be136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ab9fea8cd34265a20d859ac1bdc026, user_id=7b355d17e49a4a04b2595ec3b0bbd7fa, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:25.379287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e198fb9-75a9-45c3-8c85-6696739be136 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:25.410241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 856/1712] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:25 2022] PUT /identity/v3/projects/22ab9fea8cd34265a20d859ac1bdc026/users/7b355d17e49a4a04b2595ec3b0bbd7fa/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:25.413057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e198fb9-75a9-45c3-8c85-6696739be136 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:25.440788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb882343-5c7d-4720-ad31-9dfe6e928eb3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:25.544062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818e31d-5581-4417-a70a-b898126d0676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:25.547599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818e31d-5581-4417-a70a-b898126d0676 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:25.548391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818e31d-5581-4417-a70a-b898126d0676 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:25.548840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818e31d-5581-4417-a70a-b898126d0676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:25.585742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e198fb9-75a9-45c3-8c85-6696739be136 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:25.595467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffa512-8476-4bf7-9ff4-3c86160052e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:25.596803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffa512-8476-4bf7-9ff4-3c86160052e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:25.596803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffa512-8476-4bf7-9ff4-3c86160052e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:25.597042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffa512-8476-4bf7-9ff4-3c86160052e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:25.598075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffa512-8476-4bf7-9ff4-3c86160052e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:25.602434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffa512-8476-4bf7-9ff4-3c86160052e8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:25.627329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 857/1713] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:25.628647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb426-8b54-4682-aa6d-daead311dd2f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:25.628936 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb426-8b54-4682-aa6d-daead311dd2f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:25.629214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb426-8b54-4682-aa6d-daead311dd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:25.710738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d1beb426-8b54-4682-aa6d-daead311dd2f None None] MFA Rules not processed for user `7b355d17e49a4a04b2595ec3b0bbd7fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:25.712160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d1beb426-8b54-4682-aa6d-daead311dd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:25.764682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818e31d-5581-4417-a70a-b898126d0676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:25.767535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818e31d-5581-4417-a70a-b898126d0676 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:25.820179 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 857/1714] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:25.825712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818e31d-5581-4417-a70a-b898126d0676 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:25.858425 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818e31d-5581-4417-a70a-b898126d0676 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:25.868656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 858/1715] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:25.870904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab99560-b524-4afb-9c62-1d4495df9ac4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:25.870904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab99560-b524-4afb-9c62-1d4495df9ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:25.870904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab99560-b524-4afb-9c62-1d4495df9ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:25.882935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a3ea2-e2d7-4087-a142-e1ebd7048fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:25.885578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a3ea2-e2d7-4087-a142-e1ebd7048fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:25.889776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a3ea2-e2d7-4087-a142-e1ebd7048fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:25.891957 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a3ea2-e2d7-4087-a142-e1ebd7048fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:25.895858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a3ea2-e2d7-4087-a142-e1ebd7048fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:25.903431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a3ea2-e2d7-4087-a142-e1ebd7048fc3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:25.946372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 858/1716] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:25.949644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a3ea2-e2d7-4087-a142-e1ebd7048fc3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:25.958313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5ab99560-b524-4afb-9c62-1d4495df9ac4 None None] MFA Rules not processed for user `1ec24729b04947b7b05cc6dcd787eb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:25.959571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5ab99560-b524-4afb-9c62-1d4495df9ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:25.980735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a3ea2-e2d7-4087-a142-e1ebd7048fc3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:26.005979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc0cd4-9c1d-4a19-9cea-6188c99f601d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:26.007597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc0cd4-9c1d-4a19-9cea-6188c99f601d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:26.007988 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc0cd4-9c1d-4a19-9cea-6188c99f601d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:26.008370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc0cd4-9c1d-4a19-9cea-6188c99f601d admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:26.009131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc0cd4-9c1d-4a19-9cea-6188c99f601d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:26.011778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc0cd4-9c1d-4a19-9cea-6188c99f601d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:26.082645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:26.086684 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 859/1717] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:25 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:26.089260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc0cd4-9c1d-4a19-9cea-6188c99f601d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:26.103895 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:26.135317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 859/1718] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:26.136180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab99560-b524-4afb-9c62-1d4495df9ac4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:26.180046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab99560-b524-4afb-9c62-1d4495df9ac4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:26.276912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc0cd4-9c1d-4a19-9cea-6188c99f601d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:26.289137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb430c-cd9b-4547-bc83-3f7de40a2623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:26.292908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb430c-cd9b-4547-bc83-3f7de40a2623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:26.293829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb430c-cd9b-4547-bc83-3f7de40a2623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:26.294259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb430c-cd9b-4547-bc83-3f7de40a2623 admin admin] PATH_INFO: `/v3/projects/d577cf46559f4789b1d96c5eb5c80456/users/3c0d95fdb24c4fd0aeb0cd707eaa5bee/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:26.330239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb430c-cd9b-4547-bc83-3f7de40a2623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d577cf46559f4789b1d96c5eb5c80456, user_id=3c0d95fdb24c4fd0aeb0cd707eaa5bee, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:26.350611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb430c-cd9b-4547-bc83-3f7de40a2623 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:26.393475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 860/1719] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:26 2022] PUT /identity/v3/projects/d577cf46559f4789b1d96c5eb5c80456/users/3c0d95fdb24c4fd0aeb0cd707eaa5bee/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:26.403744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb430c-cd9b-4547-bc83-3f7de40a2623 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:26.407245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e679763-115c-43ba-80ec-5dfb3a4d1753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:26.409689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-27fb430c-cd9b-4547-bc83-3f7de40a2623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:26.411007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e679763-115c-43ba-80ec-5dfb3a4d1753 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:26.411868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e679763-115c-43ba-80ec-5dfb3a4d1753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:26.415432 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e679763-115c-43ba-80ec-5dfb3a4d1753 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:26.417377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e679763-115c-43ba-80ec-5dfb3a4d1753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:26.423646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e679763-115c-43ba-80ec-5dfb3a4d1753 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:26.495989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 860/1720] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:26 2022] POST /identity/v3/users => generated 289 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:26.495989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da95fa-8fa0-4e8c-8b1b-1949744e3631 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:26.495989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da95fa-8fa0-4e8c-8b1b-1949744e3631 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:26.495989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da95fa-8fa0-4e8c-8b1b-1949744e3631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:26.599243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-69da95fa-8fa0-4e8c-8b1b-1949744e3631 None None] MFA Rules not processed for user `3c0d95fdb24c4fd0aeb0cd707eaa5bee`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:26.601378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-69da95fa-8fa0-4e8c-8b1b-1949744e3631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:26.742589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb430c-cd9b-4547-bc83-3f7de40a2623 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:26.768679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ebcad-0663-4e02-bbf9-b067b29c187b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:26.769471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ebcad-0663-4e02-bbf9-b067b29c187b service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:26.769848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ebcad-0663-4e02-bbf9-b067b29c187b service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:26.770253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ebcad-0663-4e02-bbf9-b067b29c187b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:26.856141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 861/1721] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:26.856871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da95fa-8fa0-4e8c-8b1b-1949744e3631 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:26.950687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ebcad-0663-4e02-bbf9-b067b29c187b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:26.964283 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ebcad-0663-4e02-bbf9-b067b29c187b service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:27.006963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 861/1722] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:27.008461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ebcad-0663-4e02-bbf9-b067b29c187b service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:27.012058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da95fa-8fa0-4e8c-8b1b-1949744e3631 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:27.018593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487d4b3-e122-429b-9f89-ee84272c202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:27.019233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487d4b3-e122-429b-9f89-ee84272c202d admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:27.019518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487d4b3-e122-429b-9f89-ee84272c202d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:27.019795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487d4b3-e122-429b-9f89-ee84272c202d admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:27.020450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487d4b3-e122-429b-9f89-ee84272c202d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:27.023977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487d4b3-e122-429b-9f89-ee84272c202d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:27.040079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ebcad-0663-4e02-bbf9-b067b29c187b service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:27.052330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 862/1723] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:27.054754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487d4b3-e122-429b-9f89-ee84272c202d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:27.074095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:27.074766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:27.075052 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:27.075330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] PATH_INFO: `/v3/users/609d915f4d014d51aa1d8182df46ddaf` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:27.075909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:27.077489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=609d915f4d014d51aa1d8182df46ddaf)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:27.079495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:27.080336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487d4b3-e122-429b-9f89-ee84272c202d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:27.099455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e54083-c48d-414f-b31c-a2ba91f8157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:27.100101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e54083-c48d-414f-b31c-a2ba91f8157f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:27.100370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e54083-c48d-414f-b31c-a2ba91f8157f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:27.100647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e54083-c48d-414f-b31c-a2ba91f8157f admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:27.101287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e54083-c48d-414f-b31c-a2ba91f8157f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:27.103229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e54083-c48d-414f-b31c-a2ba91f8157f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:27.122687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:27.172268 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 863/1724] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:27 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:27.173584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5baae2-b109-4459-bc3b-6fe625d36d20 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:27.173913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5baae2-b109-4459-bc3b-6fe625d36d20 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:27.174200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5baae2-b109-4459-bc3b-6fe625d36d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:27.190605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '609d915f4d014d51aa1d8182df46ddaf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:27.192885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '609d915f4d014d51aa1d8182df46ddaf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:27.216590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '609d915f4d014d51aa1d8182df46ddaf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:27.217802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '609d915f4d014d51aa1d8182df46ddaf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:27.266987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '609d915f4d014d51aa1d8182df46ddaf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:27.276932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-de5baae2-b109-4459-bc3b-6fe625d36d20 None None] MFA Rules not processed for user `9fb2f0174dd04c0fb2aa0ac142ffce95`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:27.287615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-de5baae2-b109-4459-bc3b-6fe625d36d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:27.342654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 862/1725] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:27 2022] DELETE /identity/v3/users/609d915f4d014d51aa1d8182df46ddaf => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:27.343736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:27.350890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:27.470198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 864/1726] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:27.481171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5baae2-b109-4459-bc3b-6fe625d36d20 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:27.488483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-de5baae2-b109-4459-bc3b-6fe625d36d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:27.557699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37a7b4-1ccd-4fef-8b44-78ce42cf22e6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:27.581391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb257b-1db8-452f-9fff-cf66f5fbedd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:27.582844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb257b-1db8-452f-9fff-cf66f5fbedd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:27.583275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb257b-1db8-452f-9fff-cf66f5fbedd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:27.583697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb257b-1db8-452f-9fff-cf66f5fbedd1 admin admin] PATH_INFO: `/v3/projects/3a75e0c8c75c4b539345332c77539066/users/bd2e33a497614e49a63e996b4681bee2/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:27.646903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb257b-1db8-452f-9fff-cf66f5fbedd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a75e0c8c75c4b539345332c77539066, user_id=bd2e33a497614e49a63e996b4681bee2, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:27.649897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb257b-1db8-452f-9fff-cf66f5fbedd1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:27.691284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5baae2-b109-4459-bc3b-6fe625d36d20 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:27.700005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 863/1727] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:27 2022] PUT /identity/v3/projects/3a75e0c8c75c4b539345332c77539066/users/bd2e33a497614e49a63e996b4681bee2/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:27.705229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd420c4-228d-42e9-a51c-c0b4b57b21b1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:27.711375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd420c4-228d-42e9-a51c-c0b4b57b21b1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:27.711950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd420c4-228d-42e9-a51c-c0b4b57b21b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:27.799374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cbd420c4-228d-42e9-a51c-c0b4b57b21b1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:27.805604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cbd420c4-228d-42e9-a51c-c0b4b57b21b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:27.879884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46369f97-a44c-4017-8a44-22d8a1024609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:27.880758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46369f97-a44c-4017-8a44-22d8a1024609 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:27.881187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46369f97-a44c-4017-8a44-22d8a1024609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:27.881594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46369f97-a44c-4017-8a44-22d8a1024609 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:27.883783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46369f97-a44c-4017-8a44-22d8a1024609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:27.886001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46369f97-a44c-4017-8a44-22d8a1024609 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:27.957785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 865/1728] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:27 2022] POST /identity/v3/users => generated 296 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:27.957785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46369f97-a44c-4017-8a44-22d8a1024609 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:27.974932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-46369f97-a44c-4017-8a44-22d8a1024609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:28.019081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 864/1729] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:28.020109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd420c4-228d-42e9-a51c-c0b4b57b21b1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:28.021213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:28.054418 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd420c4-228d-42e9-a51c-c0b4b57b21b1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:28.071410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca479a-08d6-4074-8193-81cf0c589ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:28.076706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca479a-08d6-4074-8193-81cf0c589ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:28.076706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca479a-08d6-4074-8193-81cf0c589ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:28.076902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca479a-08d6-4074-8193-81cf0c589ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:28.078388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca479a-08d6-4074-8193-81cf0c589ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:28.083686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca479a-08d6-4074-8193-81cf0c589ea9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:28.115266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 865/1730] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:28.115266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca479a-08d6-4074-8193-81cf0c589ea9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:28.128574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca479a-08d6-4074-8193-81cf0c589ea9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:28.146792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba620c-bc3a-4835-babd-cd6416e422a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:28.147583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba620c-bc3a-4835-babd-cd6416e422a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:28.148203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba620c-bc3a-4835-babd-cd6416e422a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:28.148646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba620c-bc3a-4835-babd-cd6416e422a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:28.151234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba620c-bc3a-4835-babd-cd6416e422a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:28.158662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba620c-bc3a-4835-babd-cd6416e422a4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:28.161271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:28.227361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 866/1731] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:28.229117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba620c-bc3a-4835-babd-cd6416e422a4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:28.231161 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-aeba620c-bc3a-4835-babd-cd6416e422a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:28.234426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46369f97-a44c-4017-8a44-22d8a1024609 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:28.252134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cb5ca-c6ab-475f-9445-2a75b133b250 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:28.253253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cb5ca-c6ab-475f-9445-2a75b133b250 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:28.253462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cb5ca-c6ab-475f-9445-2a75b133b250 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:28.253737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cb5ca-c6ab-475f-9445-2a75b133b250 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:28.428952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cb5ca-c6ab-475f-9445-2a75b133b250 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:28.431124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cb5ca-c6ab-475f-9445-2a75b133b250 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:28.447941 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 866/1732] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:28.448792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cb5ca-c6ab-475f-9445-2a75b133b250 service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:28.483491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cb5ca-c6ab-475f-9445-2a75b133b250 service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:28.512295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a85359-8eea-4534-a5ef-12dbc6a8d2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:28.512998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a85359-8eea-4534-a5ef-12dbc6a8d2bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:28.513279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a85359-8eea-4534-a5ef-12dbc6a8d2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:28.513553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a85359-8eea-4534-a5ef-12dbc6a8d2bd admin admin] PATH_INFO: `/v3/projects/3a75e0c8c75c4b539345332c77539066/users/bd2e33a497614e49a63e996b4681bee2/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:28.518917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a85359-8eea-4534-a5ef-12dbc6a8d2bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a75e0c8c75c4b539345332c77539066, user_id=bd2e33a497614e49a63e996b4681bee2, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:28.527483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a85359-8eea-4534-a5ef-12dbc6a8d2bd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:28.539673 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba620c-bc3a-4835-babd-cd6416e422a4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:28.554801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 867/1733] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:28 2022] PUT /identity/v3/projects/3a75e0c8c75c4b539345332c77539066/users/bd2e33a497614e49a63e996b4681bee2/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:28.570746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a85359-8eea-4534-a5ef-12dbc6a8d2bd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:28.759012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d4431-a1aa-4f20-a248-6708ff50a75c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:28.767378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d4431-a1aa-4f20-a248-6708ff50a75c service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:28.767671 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d4431-a1aa-4f20-a248-6708ff50a75c service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:28.767950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d4431-a1aa-4f20-a248-6708ff50a75c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:28.852839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a85359-8eea-4534-a5ef-12dbc6a8d2bd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:28.872535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12b949-9c22-4be2-9255-e208f49592d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:28.876264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12b949-9c22-4be2-9255-e208f49592d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:28.876563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12b949-9c22-4be2-9255-e208f49592d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:28.876845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12b949-9c22-4be2-9255-e208f49592d1 admin admin] PATH_INFO: `/v3/projects/300991d9af2a45a8935d4ba27a111270/users/6d3bfaf67796491ab87643a3f99a1d2a/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:28.934591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12b949-9c22-4be2-9255-e208f49592d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=300991d9af2a45a8935d4ba27a111270, user_id=6d3bfaf67796491ab87643a3f99a1d2a, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:28.942258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12b949-9c22-4be2-9255-e208f49592d1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:28.995094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 868/1734] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:28 2022] PUT /identity/v3/projects/300991d9af2a45a8935d4ba27a111270/users/6d3bfaf67796491ab87643a3f99a1d2a/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:28.995621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791d96e-f972-4004-b600-60b46efad286 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:28.995879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d4431-a1aa-4f20-a248-6708ff50a75c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:28.996023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791d96e-f972-4004-b600-60b46efad286 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:28.996181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791d96e-f972-4004-b600-60b46efad286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:28.997355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d4431-a1aa-4f20-a248-6708ff50a75c service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:29.023357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 867/1735] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:29.026017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5537cd-0ac0-488e-bcac-dc0383650c49 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:29.026017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5537cd-0ac0-488e-bcac-dc0383650c49 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:29.026017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5537cd-0ac0-488e-bcac-dc0383650c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:29.080882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1791d96e-f972-4004-b600-60b46efad286 None None] MFA Rules not processed for user `e7527106829643448f153f9431f233f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:29.082756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1791d96e-f972-4004-b600-60b46efad286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:29.097577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:29.142994 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4b5537cd-0ac0-488e-bcac-dc0383650c49 None None] MFA Rules not processed for user `bd2e33a497614e49a63e996b4681bee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:29.144283 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4b5537cd-0ac0-488e-bcac-dc0383650c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:29.264021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 869/1736] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:29.265491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791d96e-f972-4004-b600-60b46efad286 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:29.293470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:29.341571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 868/1737] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:29.346087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d16c4f-7e07-49df-84aa-4e828bb997ff None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:29.346383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d16c4f-7e07-49df-84aa-4e828bb997ff None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:29.346704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d16c4f-7e07-49df-84aa-4e828bb997ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:29.409000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-66d16c4f-7e07-49df-84aa-4e828bb997ff None None] MFA Rules not processed for user `e7527106829643448f153f9431f233f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:29.414556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-66d16c4f-7e07-49df-84aa-4e828bb997ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:29.452430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 869/1738] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:29.457820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d16c4f-7e07-49df-84aa-4e828bb997ff None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:29.459278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-66d16c4f-7e07-49df-84aa-4e828bb997ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:29.506393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791d96e-f972-4004-b600-60b46efad286 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:29.523461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ff4dd-9c7e-4439-a0d7-c3348d02f9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:29.530235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ff4dd-9c7e-4439-a0d7-c3348d02f9cd admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:29.531928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ff4dd-9c7e-4439-a0d7-c3348d02f9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:29.532350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ff4dd-9c7e-4439-a0d7-c3348d02f9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:29.550046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ff4dd-9c7e-4439-a0d7-c3348d02f9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:29.550046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ff4dd-9c7e-4439-a0d7-c3348d02f9cd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:29.590672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 870/1739] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:29.591644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ff4dd-9c7e-4439-a0d7-c3348d02f9cd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:29.600894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9d3ff4dd-9c7e-4439-a0d7-c3348d02f9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:29.684916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d16c4f-7e07-49df-84aa-4e828bb997ff None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:29.695550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1a04-2e8f-428a-bae8-a47628f493f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:29.698706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1a04-2e8f-428a-bae8-a47628f493f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:29.700807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1a04-2e8f-428a-bae8-a47628f493f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:29.703035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1a04-2e8f-428a-bae8-a47628f493f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:29.732604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1a04-2e8f-428a-bae8-a47628f493f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:29.736339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1a04-2e8f-428a-bae8-a47628f493f9 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:29.779295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 870/1740] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:29.784317 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1a04-2e8f-428a-bae8-a47628f493f9 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:29.863090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1a04-2e8f-428a-bae8-a47628f493f9 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:29.903071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfee88-f8cc-4d17-9a85-5eb6a3ec435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:29.905135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfee88-f8cc-4d17-9a85-5eb6a3ec435d admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:29.908204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfee88-f8cc-4d17-9a85-5eb6a3ec435d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:29.910744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfee88-f8cc-4d17-9a85-5eb6a3ec435d admin admin] PATH_INFO: `/v3/projects/300991d9af2a45a8935d4ba27a111270/users/6d3bfaf67796491ab87643a3f99a1d2a/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:29.917686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfee88-f8cc-4d17-9a85-5eb6a3ec435d admin admin] RBAC: Authorizing `identity:create_grant(project_id=300991d9af2a45a8935d4ba27a111270, user_id=6d3bfaf67796491ab87643a3f99a1d2a, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:29.926206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ff4dd-9c7e-4439-a0d7-c3348d02f9cd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:29.930410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfee88-f8cc-4d17-9a85-5eb6a3ec435d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:29.954225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d696e40-36a3-4d07-bc67-891d54f5372e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:29.959129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d696e40-36a3-4d07-bc67-891d54f5372e service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:29.962223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d696e40-36a3-4d07-bc67-891d54f5372e service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:29.962509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d696e40-36a3-4d07-bc67-891d54f5372e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:30.018797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 871/1741] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:29 2022] PUT /identity/v3/projects/300991d9af2a45a8935d4ba27a111270/users/6d3bfaf67796491ab87643a3f99a1d2a/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:30.027338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce2f77-12ae-4a1e-80ea-3a3d56f4a917 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:30.027723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce2f77-12ae-4a1e-80ea-3a3d56f4a917 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:30.027901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce2f77-12ae-4a1e-80ea-3a3d56f4a917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:30.029655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d696e40-36a3-4d07-bc67-891d54f5372e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:30.036263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d696e40-36a3-4d07-bc67-891d54f5372e service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:30.068368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 871/1742] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:30.071096 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d696e40-36a3-4d07-bc67-891d54f5372e service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:30.100859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d696e40-36a3-4d07-bc67-891d54f5372e service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:30.152509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-afce2f77-12ae-4a1e-80ea-3a3d56f4a917 None None] MFA Rules not processed for user `6d3bfaf67796491ab87643a3f99a1d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:30.161988 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-afce2f77-12ae-4a1e-80ea-3a3d56f4a917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:30.200031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:30.269053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee98691-def6-49cb-933b-94dabdb95f83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:30.271137 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98691-def6-49cb-933b-94dabdb95f83 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:30.272171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98691-def6-49cb-933b-94dabdb95f83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:30.272942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98691-def6-49cb-933b-94dabdb95f83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:30.277043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2ee98691-def6-49cb-933b-94dabdb95f83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:30.379139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 872/1743] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:30.395103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce2f77-12ae-4a1e-80ea-3a3d56f4a917 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:30.397870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-afce2f77-12ae-4a1e-80ea-3a3d56f4a917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:30.440608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce2f77-12ae-4a1e-80ea-3a3d56f4a917 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:30.454816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def6acb-fd87-49cb-8e12-fcefcb3631d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:30.458794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def6acb-fd87-49cb-8e12-fcefcb3631d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:30.459117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def6acb-fd87-49cb-8e12-fcefcb3631d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:30.459864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def6acb-fd87-49cb-8e12-fcefcb3631d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:30.515519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def6acb-fd87-49cb-8e12-fcefcb3631d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:30.518614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def6acb-fd87-49cb-8e12-fcefcb3631d8 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:30.553707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee98691-def6-49cb-933b-94dabdb95f83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:30.555222 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee98691-def6-49cb-933b-94dabdb95f83 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:30.566175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 873/1744] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:27:30.593712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 872/1745] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:30.747361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def6acb-fd87-49cb-8e12-fcefcb3631d8 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:30.758830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def6acb-fd87-49cb-8e12-fcefcb3631d8 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:30.769551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ff741-f15d-45dd-981b-903082e10197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:30.770202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ff741-f15d-45dd-981b-903082e10197 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:30.770480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ff741-f15d-45dd-981b-903082e10197 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:30.770754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ff741-f15d-45dd-981b-903082e10197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:30.782080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1b3ff741-f15d-45dd-981b-903082e10197 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:30.899578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:30.930233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ff741-f15d-45dd-981b-903082e10197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:30.932281 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ff741-f15d-45dd-981b-903082e10197 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:30.960577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 874/1746] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:31.383618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a218ce4-5440-44d5-b6d0-c9e566dcd31a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:31.384242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a218ce4-5440-44d5-b6d0-c9e566dcd31a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:31.384721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a218ce4-5440-44d5-b6d0-c9e566dcd31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:31.498680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2a218ce4-5440-44d5-b6d0-c9e566dcd31a None None] MFA Rules not processed for user `75d4379ec87240389ce9c077886ab472`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:31.509234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2a218ce4-5440-44d5-b6d0-c9e566dcd31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:31.777176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 873/1747] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:31.792049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777527e-946e-46f6-aa59-ef2b298e7587 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:31.793069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777527e-946e-46f6-aa59-ef2b298e7587 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:31.793486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777527e-946e-46f6-aa59-ef2b298e7587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:31.846773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3777527e-946e-46f6-aa59-ef2b298e7587 None None] MFA Rules not processed for user `75d4379ec87240389ce9c077886ab472`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:31.850774 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3777527e-946e-46f6-aa59-ef2b298e7587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:31.875989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 875/1748] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:31.889143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a218ce4-5440-44d5-b6d0-c9e566dcd31a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:32.080080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a218ce4-5440-44d5-b6d0-c9e566dcd31a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:32.094614 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3b0f8-f050-4635-9bc7-bd59984e85d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:32.096830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3b0f8-f050-4635-9bc7-bd59984e85d1 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:32.097169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3b0f8-f050-4635-9bc7-bd59984e85d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:32.097448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3b0f8-f050-4635-9bc7-bd59984e85d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:32.114659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:32.118910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3b0f8-f050-4635-9bc7-bd59984e85d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:32.124371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3b0f8-f050-4635-9bc7-bd59984e85d1 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:32.158086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 874/1749] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:32.819290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777527e-946e-46f6-aa59-ef2b298e7587 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:32.820871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3777527e-946e-46f6-aa59-ef2b298e7587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:32.927366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fefdef-bd74-402b-b4f7-51d6ad921694 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:32.927894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fefdef-bd74-402b-b4f7-51d6ad921694 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:32.927980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fefdef-bd74-402b-b4f7-51d6ad921694 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:32.929548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 875/1750] 10.222.0.18 () {54 vars in 912 bytes} [Wed Sep 21 05:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:27:32.950785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a1e4c-5dd9-4168-a113-4d98229d4f52 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:32.950978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a1e4c-5dd9-4168-a113-4d98229d4f52 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:32.951247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a1e4c-5dd9-4168-a113-4d98229d4f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:32.982426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777527e-946e-46f6-aa59-ef2b298e7587 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:33.008067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:33.008780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:33.009225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:33.009730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] PATH_INFO: `/v3/projects/089de397f5ff4e26b191403a0341f65e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:33.012667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=089de397f5ff4e26b191403a0341f65e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:33.014984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:33.070814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-da5a1e4c-5dd9-4168-a113-4d98229d4f52 None None] MFA Rules not processed for user `c07403facfaf4c71ab5324af6c44f52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:33.072162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-da5a1e4c-5dd9-4168-a113-4d98229d4f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:33.170506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '089de397f5ff4e26b191403a0341f65e', 'request_id': 'req-5981b129-8bf8-4ba5-aa20-8fbec1601984'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.185599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '089de397f5ff4e26b191403a0341f65e', 'request_id': 'req-5981b129-8bf8-4ba5-aa20-8fbec1601984'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.201576 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '089de397f5ff4e26b191403a0341f65e', 'request_id': 'req-5981b129-8bf8-4ba5-aa20-8fbec1601984'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.224388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] The token cache is being invalidate because project 089de397f5ff4e26b191403a0341f65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:33.225095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.226121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.235008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 876/1751] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:32 2022] DELETE /identity/v3/projects/089de397f5ff4e26b191403a0341f65e => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:33.238959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:33.244654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:33.255396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 876/1752] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:27:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:33.449271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981b129-8bf8-4ba5-aa20-8fbec1601984 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:33.466550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:33.467508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:33.467933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:33.468416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] PATH_INFO: `/v3/users/9c803c822bb04ab89be9440e4f6ba99f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:33.470133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c803c822bb04ab89be9440e4f6ba99f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:33.476155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:33.484704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:33.568560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a1e4c-5dd9-4168-a113-4d98229d4f52 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:33.570169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-da5a1e4c-5dd9-4168-a113-4d98229d4f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:33.584048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c803c822bb04ab89be9440e4f6ba99f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.589517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c803c822bb04ab89be9440e4f6ba99f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.598484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a1e4c-5dd9-4168-a113-4d98229d4f52 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:33.609983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c803c822bb04ab89be9440e4f6ba99f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.612099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c803c822bb04ab89be9440e4f6ba99f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.634720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7cad8822-4b80-4827-ace2-597d6b759d07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c803c822bb04ab89be9440e4f6ba99f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:33.700121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 877/1753] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:33 2022] DELETE /identity/v3/users/9c803c822bb04ab89be9440e4f6ba99f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:33.762474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98deffb1-8680-4d86-b0d9-0195af0dbc90 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:33.762935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98deffb1-8680-4d86-b0d9-0195af0dbc90 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:33.763305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98deffb1-8680-4d86-b0d9-0195af0dbc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:33.806645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf4e2a-50a1-4a70-8c50-587bfc707602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:33.809265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4e2a-50a1-4a70-8c50-587bfc707602 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:33.819723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4e2a-50a1-4a70-8c50-587bfc707602 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:33.822920 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4e2a-50a1-4a70-8c50-587bfc707602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:33.827479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bedf4e2a-50a1-4a70-8c50-587bfc707602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:33.836471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-98deffb1-8680-4d86-b0d9-0195af0dbc90 None None] MFA Rules not processed for user `1ec24729b04947b7b05cc6dcd787eb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:33.839067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-98deffb1-8680-4d86-b0d9-0195af0dbc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:33.859181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf4e2a-50a1-4a70-8c50-587bfc707602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:33.860878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf4e2a-50a1-4a70-8c50-587bfc707602 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:33.873273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 877/1754] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:27:33.879953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad67ae8-6c51-47db-9ea3-7de4c8f66668 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:33.880551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad67ae8-6c51-47db-9ea3-7de4c8f66668 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:33.880994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad67ae8-6c51-47db-9ea3-7de4c8f66668 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:33.887525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 878/1755] 10.222.0.18 () {54 vars in 912 bytes} [Wed Sep 21 05:27:33 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:27:33.889100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d91bdd-635e-4e6d-bd1b-0ef40f25b71d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:33.889491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d91bdd-635e-4e6d-bd1b-0ef40f25b71d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:33.889778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d91bdd-635e-4e6d-bd1b-0ef40f25b71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:33.942798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-02d91bdd-635e-4e6d-bd1b-0ef40f25b71d None None] MFA Rules not processed for user `c07403facfaf4c71ab5324af6c44f52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:33.950646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-02d91bdd-635e-4e6d-bd1b-0ef40f25b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:34.078932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 878/1756] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:34.087800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 879/1757] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:27:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:34.107752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98deffb1-8680-4d86-b0d9-0195af0dbc90 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:34.110117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-98deffb1-8680-4d86-b0d9-0195af0dbc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:34.125082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98deffb1-8680-4d86-b0d9-0195af0dbc90 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:34.129472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b314-db67-4efb-a764-fb9c3a3623eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:34.131277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b314-db67-4efb-a764-fb9c3a3623eb service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:34.131277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b314-db67-4efb-a764-fb9c3a3623eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:34.131277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b314-db67-4efb-a764-fb9c3a3623eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:34.140923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7b314-db67-4efb-a764-fb9c3a3623eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:34.142595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7b314-db67-4efb-a764-fb9c3a3623eb service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:34.153653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 879/1758] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:34.482673 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388a4dd-ab49-4f0f-bac2-13ea0d1b2a4d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:34.482968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388a4dd-ab49-4f0f-bac2-13ea0d1b2a4d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:34.483509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388a4dd-ab49-4f0f-bac2-13ea0d1b2a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:34.555149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5388a4dd-ab49-4f0f-bac2-13ea0d1b2a4d None None] MFA Rules not processed for user `9fb2f0174dd04c0fb2aa0ac142ffce95`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:34.557014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5388a4dd-ab49-4f0f-bac2-13ea0d1b2a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:34.710965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:34.764719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 880/1759] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:34.822755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b314-db67-4efb-a764-fb9c3a3623eb service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:34.824306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-58f7b314-db67-4efb-a764-fb9c3a3623eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:34.865104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388a4dd-ab49-4f0f-bac2-13ea0d1b2a4d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:34.867266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5388a4dd-ab49-4f0f-bac2-13ea0d1b2a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:35.093485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b314-db67-4efb-a764-fb9c3a3623eb service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:35.103993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:35.104812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:35.105187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:35.105468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] PATH_INFO: `/v3/projects/2afd0f4e012848349b699e0a0ebc2956` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:35.107413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2afd0f4e012848349b699e0a0ebc2956)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:35.110565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:35.131580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388a4dd-ab49-4f0f-bac2-13ea0d1b2a4d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:35.148036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893bbc1-4337-45b9-b25e-87236baa6b27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:35.148563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893bbc1-4337-45b9-b25e-87236baa6b27 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:35.148910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893bbc1-4337-45b9-b25e-87236baa6b27 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:35.149190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893bbc1-4337-45b9-b25e-87236baa6b27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:35.153738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9893bbc1-4337-45b9-b25e-87236baa6b27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:35.168319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:35.194313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893bbc1-4337-45b9-b25e-87236baa6b27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:35.200878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893bbc1-4337-45b9-b25e-87236baa6b27 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:35.253387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 881/1760] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:27:35.282681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2afd0f4e012848349b699e0a0ebc2956', 'request_id': 'req-21288b4c-00d6-4766-aa28-ef8a9db2bf94'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:35.306389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2afd0f4e012848349b699e0a0ebc2956', 'request_id': 'req-21288b4c-00d6-4766-aa28-ef8a9db2bf94'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:35.325391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2afd0f4e012848349b699e0a0ebc2956', 'request_id': 'req-21288b4c-00d6-4766-aa28-ef8a9db2bf94'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:35.352829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] The token cache is being invalidate because project 2afd0f4e012848349b699e0a0ebc2956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:35.353274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:35.357866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21288b4c-00d6-4766-aa28-ef8a9db2bf94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:35.366537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 880/1761] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:34 2022] DELETE /identity/v3/projects/2afd0f4e012848349b699e0a0ebc2956 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:35.371307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9624-9008-4451-9a46-6051db26b4ab None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:35.371599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9624-9008-4451-9a46-6051db26b4ab None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:35.371880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9624-9008-4451-9a46-6051db26b4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:35.443852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-652a9624-9008-4451-9a46-6051db26b4ab None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:35.445109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-652a9624-9008-4451-9a46-6051db26b4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:35.762619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 882/1762] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:35.770059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cc721-fcc4-41d7-9a84-e8043a41d18a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:35.771695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cc721-fcc4-41d7-9a84-e8043a41d18a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:35.772208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cc721-fcc4-41d7-9a84-e8043a41d18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:35.830314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ab3cc721-fcc4-41d7-9a84-e8043a41d18a None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:35.832423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ab3cc721-fcc4-41d7-9a84-e8043a41d18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:35.854416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 881/1763] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:35.855284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a9624-9008-4451-9a46-6051db26b4ab None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:35.866156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a9624-9008-4451-9a46-6051db26b4ab None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:35.880229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b30fe0-e174-4f70-87fa-e110443eb197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:35.882505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b30fe0-e174-4f70-87fa-e110443eb197 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:35.883106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b30fe0-e174-4f70-87fa-e110443eb197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:35.884183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b30fe0-e174-4f70-87fa-e110443eb197 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:35.891095 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b30fe0-e174-4f70-87fa-e110443eb197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:35.891250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b30fe0-e174-4f70-87fa-e110443eb197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:35.897723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b30fe0-e174-4f70-87fa-e110443eb197 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:35.952575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 883/1764] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:35.956840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3cc721-fcc4-41d7-9a84-e8043a41d18a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:35.971844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3cc721-fcc4-41d7-9a84-e8043a41d18a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:35.987729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9638587-6a59-49b1-9c26-538d16bccf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:35.988365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9638587-6a59-49b1-9c26-538d16bccf04 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:35.988746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9638587-6a59-49b1-9c26-538d16bccf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:35.989029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9638587-6a59-49b1-9c26-538d16bccf04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:35.989876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9638587-6a59-49b1-9c26-538d16bccf04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:35.991802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9638587-6a59-49b1-9c26-538d16bccf04 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:36.044870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 882/1765] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:35 2022] POST /identity/v3/projects => generated 369 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:36.049083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b30fe0-e174-4f70-87fa-e110443eb197 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:36.065395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b30fe0-e174-4f70-87fa-e110443eb197 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:36.142490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fce58-6e44-40ae-b63f-77d795b6e00d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:36.143829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fce58-6e44-40ae-b63f-77d795b6e00d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:36.145704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fce58-6e44-40ae-b63f-77d795b6e00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:36.185478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-323fce58-6e44-40ae-b63f-77d795b6e00d None None] MFA Rules not processed for user `e7527106829643448f153f9431f233f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:36.189772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-323fce58-6e44-40ae-b63f-77d795b6e00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:36.298888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa685f-fbc6-4568-9ab4-dbe3c4c9d666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:36.299536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa685f-fbc6-4568-9ab4-dbe3c4c9d666 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:36.299914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa685f-fbc6-4568-9ab4-dbe3c4c9d666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:36.300196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa685f-fbc6-4568-9ab4-dbe3c4c9d666 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:36.300843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa685f-fbc6-4568-9ab4-dbe3c4c9d666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:36.313363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa685f-fbc6-4568-9ab4-dbe3c4c9d666 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:36.365278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:36.416430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 884/1766] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:36 2022] POST /identity/v3/users => generated 285 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:36.417396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa685f-fbc6-4568-9ab4-dbe3c4c9d666 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:36.452502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 883/1767] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:36.477261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa685f-fbc6-4568-9ab4-dbe3c4c9d666 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:36.496060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245db12-c16f-4a4c-ad3b-de5d8f328a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:36.497269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245db12-c16f-4a4c-ad3b-de5d8f328a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:36.499333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245db12-c16f-4a4c-ad3b-de5d8f328a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:36.503171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245db12-c16f-4a4c-ad3b-de5d8f328a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:36.504109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245db12-c16f-4a4c-ad3b-de5d8f328a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:36.513784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245db12-c16f-4a4c-ad3b-de5d8f328a22 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:36.547978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 885/1768] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:36.553060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fce58-6e44-40ae-b63f-77d795b6e00d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:36.570393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fce58-6e44-40ae-b63f-77d795b6e00d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:36.577696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25307e0-7ad3-488e-947a-b62bcc69ac3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:36.578515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25307e0-7ad3-488e-947a-b62bcc69ac3e admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:36.578918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25307e0-7ad3-488e-947a-b62bcc69ac3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:36.579218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25307e0-7ad3-488e-947a-b62bcc69ac3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:36.580115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25307e0-7ad3-488e-947a-b62bcc69ac3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:36.582139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25307e0-7ad3-488e-947a-b62bcc69ac3e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:36.626406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 884/1769] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:36.627963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245db12-c16f-4a4c-ad3b-de5d8f328a22 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:36.657195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245db12-c16f-4a4c-ad3b-de5d8f328a22 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:36.674306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db175d11-51c1-4763-8988-d8817a58e056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:36.674998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db175d11-51c1-4763-8988-d8817a58e056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:36.675274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db175d11-51c1-4763-8988-d8817a58e056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:36.675550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db175d11-51c1-4763-8988-d8817a58e056 admin admin] PATH_INFO: `/v3/projects/5edd4dfbc75d439cad4aaf15ac916464/users/55c912d5db4b4706b864ddb296f2e7e7/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:36.762724 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db175d11-51c1-4763-8988-d8817a58e056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5edd4dfbc75d439cad4aaf15ac916464, user_id=55c912d5db4b4706b864ddb296f2e7e7, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:36.767753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db175d11-51c1-4763-8988-d8817a58e056 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:36.790392 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 886/1770] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:36 2022] PUT /identity/v3/projects/5edd4dfbc75d439cad4aaf15ac916464/users/55c912d5db4b4706b864ddb296f2e7e7/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:36.802707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49947957-4f56-4b19-9d09-24df95bab5c7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:36.802992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49947957-4f56-4b19-9d09-24df95bab5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:36.803268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49947957-4f56-4b19-9d09-24df95bab5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:36.881216 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-49947957-4f56-4b19-9d09-24df95bab5c7 None None] MFA Rules not processed for user `55c912d5db4b4706b864ddb296f2e7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:36.883645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-49947957-4f56-4b19-9d09-24df95bab5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:36.987555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:37.000808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 885/1771] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:37.002097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db175d11-51c1-4763-8988-d8817a58e056 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:37.154186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db175d11-51c1-4763-8988-d8817a58e056 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:37.181971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8aaf4a-ecea-463e-966a-75d1d40fbfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:37.182906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8aaf4a-ecea-463e-966a-75d1d40fbfc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:37.183382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8aaf4a-ecea-463e-966a-75d1d40fbfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:37.183852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8aaf4a-ecea-463e-966a-75d1d40fbfc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:37.184698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8aaf4a-ecea-463e-966a-75d1d40fbfc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:37.187440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8aaf4a-ecea-463e-966a-75d1d40fbfc1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:37.256981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 887/1772] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:37 2022] POST /identity/v3/projects => generated 369 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:27:37.261322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49947957-4f56-4b19-9d09-24df95bab5c7 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:37.283734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49947957-4f56-4b19-9d09-24df95bab5c7 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:37.402120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffeb834-7bae-46b6-9c60-0a012144bbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:37.404389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffeb834-7bae-46b6-9c60-0a012144bbe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:37.405075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffeb834-7bae-46b6-9c60-0a012144bbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:37.405657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffeb834-7bae-46b6-9c60-0a012144bbe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:37.407482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffeb834-7bae-46b6-9c60-0a012144bbe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:37.410185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffeb834-7bae-46b6-9c60-0a012144bbe5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:37.457287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 886/1773] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:37 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:37.461441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8aaf4a-ecea-463e-966a-75d1d40fbfc1 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:37.486703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8aaf4a-ecea-463e-966a-75d1d40fbfc1 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:37.499941 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d7096-f90d-45cd-9228-c4454316d197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:37.502250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d7096-f90d-45cd-9228-c4454316d197 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:37.508054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d7096-f90d-45cd-9228-c4454316d197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:37.509049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d7096-f90d-45cd-9228-c4454316d197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:37.510925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d7096-f90d-45cd-9228-c4454316d197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:37.512638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d7096-f90d-45cd-9228-c4454316d197 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:37.558360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 888/1774] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:27:37.562545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffeb834-7bae-46b6-9c60-0a012144bbe5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:37.576679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffeb834-7bae-46b6-9c60-0a012144bbe5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:37.582377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b9846-080c-4abc-b8be-fa74a33beecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:37.583339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b9846-080c-4abc-b8be-fa74a33beecf admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:37.583975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b9846-080c-4abc-b8be-fa74a33beecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:37.584434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b9846-080c-4abc-b8be-fa74a33beecf admin admin] PATH_INFO: `/v3/projects/aad6b43f24ee4b1a887aaef15b4b0edb/users/54ba427c546b4453b67b742752e11642/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:37.608318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b9846-080c-4abc-b8be-fa74a33beecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad6b43f24ee4b1a887aaef15b4b0edb, user_id=54ba427c546b4453b67b742752e11642, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:37.611832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b9846-080c-4abc-b8be-fa74a33beecf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:37.630562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 887/1775] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:37 2022] PUT /identity/v3/projects/aad6b43f24ee4b1a887aaef15b4b0edb/users/54ba427c546b4453b67b742752e11642/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:37.634991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d7096-f90d-45cd-9228-c4454316d197 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:37.649925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d7096-f90d-45cd-9228-c4454316d197 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:37.715868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3341fd-1b63-44a1-9b18-7e02f8e8cd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:37.716612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3341fd-1b63-44a1-9b18-7e02f8e8cd27 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:37.716955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3341fd-1b63-44a1-9b18-7e02f8e8cd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:37.717397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3341fd-1b63-44a1-9b18-7e02f8e8cd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:37.718054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3341fd-1b63-44a1-9b18-7e02f8e8cd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:37.719955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3341fd-1b63-44a1-9b18-7e02f8e8cd27 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:37.732075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 889/1776] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:27:37.736352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b9846-080c-4abc-b8be-fa74a33beecf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:37.748163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b9846-080c-4abc-b8be-fa74a33beecf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:37.758933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8232357-483b-42fb-ab1e-3d8a549a7c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:37.759688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8232357-483b-42fb-ab1e-3d8a549a7c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:37.760036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8232357-483b-42fb-ab1e-3d8a549a7c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:37.760486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8232357-483b-42fb-ab1e-3d8a549a7c0d admin admin] PATH_INFO: `/v3/projects/aad6b43f24ee4b1a887aaef15b4b0edb/users/54ba427c546b4453b67b742752e11642/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:37.762680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8232357-483b-42fb-ab1e-3d8a549a7c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad6b43f24ee4b1a887aaef15b4b0edb, user_id=54ba427c546b4453b67b742752e11642, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:37.765402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8232357-483b-42fb-ab1e-3d8a549a7c0d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:37.780820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 888/1777] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:37 2022] PUT /identity/v3/projects/aad6b43f24ee4b1a887aaef15b4b0edb/users/54ba427c546b4453b67b742752e11642/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:27:37.793813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d389ba-6fe9-4df0-adc2-361fe72968c7 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:37.794496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d389ba-6fe9-4df0-adc2-361fe72968c7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:37.795532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d389ba-6fe9-4df0-adc2-361fe72968c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:37.830733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f5d389ba-6fe9-4df0-adc2-361fe72968c7 None None] MFA Rules not processed for user `54ba427c546b4453b67b742752e11642`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:37.832285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f5d389ba-6fe9-4df0-adc2-361fe72968c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:37.852960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:37.942955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 890/1778] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:37.960745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8232357-483b-42fb-ab1e-3d8a549a7c0d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:37.963203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b8232357-483b-42fb-ab1e-3d8a549a7c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:38.093750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8232357-483b-42fb-ab1e-3d8a549a7c0d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:38.104072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c452a1-b09e-4603-83a6-e1c510314b32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:38.107245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c452a1-b09e-4603-83a6-e1c510314b32 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:38.107557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c452a1-b09e-4603-83a6-e1c510314b32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:38.108165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c452a1-b09e-4603-83a6-e1c510314b32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:38.137194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c452a1-b09e-4603-83a6-e1c510314b32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:38.138720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c452a1-b09e-4603-83a6-e1c510314b32 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:38.163258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 889/1779] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:39.082675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d389ba-6fe9-4df0-adc2-361fe72968c7 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:39.084186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f5d389ba-6fe9-4df0-adc2-361fe72968c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:39.215840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d389ba-6fe9-4df0-adc2-361fe72968c7 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:39.224695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:39.225391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:39.225672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:39.225975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] PATH_INFO: `/v3/users/7b355d17e49a4a04b2595ec3b0bbd7fa` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:39.229090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b355d17e49a4a04b2595ec3b0bbd7fa)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:39.231964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:39.335332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b355d17e49a4a04b2595ec3b0bbd7fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:39.351743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b355d17e49a4a04b2595ec3b0bbd7fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:39.352981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b355d17e49a4a04b2595ec3b0bbd7fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:39.354400 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b355d17e49a4a04b2595ec3b0bbd7fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:39.373132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b355d17e49a4a04b2595ec3b0bbd7fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:39.457466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 891/1780] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:39 2022] DELETE /identity/v3/users/7b355d17e49a4a04b2595ec3b0bbd7fa => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:39.459579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc2194-335d-4fc4-bf71-1ebddf9759dd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:39.460096 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc2194-335d-4fc4-bf71-1ebddf9759dd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:39.460351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc2194-335d-4fc4-bf71-1ebddf9759dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:39.538970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cacc2194-335d-4fc4-bf71-1ebddf9759dd None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:39.544903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cacc2194-335d-4fc4-bf71-1ebddf9759dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:39.836225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:39.884858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 890/1781] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:40.000102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:40.003809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:40.121763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:40.179368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50efb8-9454-4b1e-8636-8527f722e6b5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:40.189743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10e55d-1fec-4d44-a3d3-e206f237015a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:40.190421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10e55d-1fec-4d44-a3d3-e206f237015a service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:40.190708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10e55d-1fec-4d44-a3d3-e206f237015a service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:40.196544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10e55d-1fec-4d44-a3d3-e206f237015a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:40.256161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10e55d-1fec-4d44-a3d3-e206f237015a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:40.258654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10e55d-1fec-4d44-a3d3-e206f237015a service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:40.287288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 892/1782] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:40.754112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f542612-312a-430f-8708-2bd1ebe7ecca None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:40.754112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f542612-312a-430f-8708-2bd1ebe7ecca None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:40.754112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f542612-312a-430f-8708-2bd1ebe7ecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:40.940842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5f542612-312a-430f-8708-2bd1ebe7ecca None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:40.942298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f542612-312a-430f-8708-2bd1ebe7ecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:41.108313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 891/1783] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:27:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:41.474790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10e55d-1fec-4d44-a3d3-e206f237015a service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:41.477320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ab10e55d-1fec-4d44-a3d3-e206f237015a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:41.520683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f542612-312a-430f-8708-2bd1ebe7ecca None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:41.523265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f542612-312a-430f-8708-2bd1ebe7ecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:41.600120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f542612-312a-430f-8708-2bd1ebe7ecca None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:41.624296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:41.624946 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:41.625248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:41.625514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] PATH_INFO: `/v3/projects/22ab9fea8cd34265a20d859ac1bdc026` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:41.643748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ab9fea8cd34265a20d859ac1bdc026)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:41.645713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:41.871864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10e55d-1fec-4d44-a3d3-e206f237015a service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:41.878585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ab9fea8cd34265a20d859ac1bdc026', 'request_id': 'req-e2831812-ef7d-46f5-9bd2-840048ba6a7c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:41.906597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ab9fea8cd34265a20d859ac1bdc026', 'request_id': 'req-e2831812-ef7d-46f5-9bd2-840048ba6a7c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:41.926914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ab9fea8cd34265a20d859ac1bdc026', 'request_id': 'req-e2831812-ef7d-46f5-9bd2-840048ba6a7c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:41.984027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] The token cache is being invalidate because project 22ab9fea8cd34265a20d859ac1bdc026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:41.984135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:41.990462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:42.006265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 892/1784] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:41 2022] DELETE /identity/v3/projects/22ab9fea8cd34265a20d859ac1bdc026 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:27:42.007255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:42.013580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:42.143965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:42.144607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:42.147499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:42.147499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:42.153498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:42.175706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:42.204973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2831812-ef7d-46f5-9bd2-840048ba6a7c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:42.219259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:42.221297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:42.221297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:42.221575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] PATH_INFO: `/v3/users/bd2e33a497614e49a63e996b4681bee2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:42.223642 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd2e33a497614e49a63e996b4681bee2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:42.237233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:42.254602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:42.256291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:42.290330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 893/1785] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:27:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 816 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:27:42.406967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd2e33a497614e49a63e996b4681bee2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:42.409572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd2e33a497614e49a63e996b4681bee2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:42.431205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd2e33a497614e49a63e996b4681bee2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:42.432008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd2e33a497614e49a63e996b4681bee2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:42.465520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd2e33a497614e49a63e996b4681bee2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:42.538969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 893/1786] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:42 2022] DELETE /identity/v3/users/bd2e33a497614e49a63e996b4681bee2 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:42.753963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:42.755495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:42.772536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:42.837122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b8a2-b985-4add-8ef6-fdcc00610100 service swift] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:42.942023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:42.944082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:43.209740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:43.210351 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:43.210489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:43.210685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] PATH_INFO: `/v3/users/55c912d5db4b4706b864ddb296f2e7e7` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:43.220375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c912d5db4b4706b864ddb296f2e7e7)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:43.222416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:43.318018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840d4cd-6e46-47dc-b9ee-1218fe68b4ea admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:43.336550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:43.337565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:43.339202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:43.339521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] PATH_INFO: `/v3/users/3c0d95fdb24c4fd0aeb0cd707eaa5bee` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:43.344168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c0d95fdb24c4fd0aeb0cd707eaa5bee)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:43.348296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:43.421320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c912d5db4b4706b864ddb296f2e7e7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.456790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c912d5db4b4706b864ddb296f2e7e7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.457968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c912d5db4b4706b864ddb296f2e7e7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.460114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c912d5db4b4706b864ddb296f2e7e7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.477483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c0d95fdb24c4fd0aeb0cd707eaa5bee'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.478965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c0d95fdb24c4fd0aeb0cd707eaa5bee'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.488356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2c3326fa-8db5-4b32-bdcd-f2ce5b858fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c912d5db4b4706b864ddb296f2e7e7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.499865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c0d95fdb24c4fd0aeb0cd707eaa5bee'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.500820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c0d95fdb24c4fd0aeb0cd707eaa5bee'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.534913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-34c353ba-81a4-4109-8752-bd0409b8ab52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c0d95fdb24c4fd0aeb0cd707eaa5bee'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:43.537187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 894/1787] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:42 2022] DELETE /identity/v3/users/55c912d5db4b4706b864ddb296f2e7e7 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:43.548617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d0c11-958c-4a9b-880a-5f13f52def87 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:43.548916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d0c11-958c-4a9b-880a-5f13f52def87 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:43.550395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d0c11-958c-4a9b-880a-5f13f52def87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:43.567981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:43.626152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 894/1788] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:42 2022] DELETE /identity/v3/users/3c0d95fdb24c4fd0aeb0cd707eaa5bee => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:43.630513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f267-bbeb-4755-8a50-e1ad510a01be None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:43.630808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f267-bbeb-4755-8a50-e1ad510a01be None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:43.631120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f267-bbeb-4755-8a50-e1ad510a01be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:43.668886 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ea9d0c11-958c-4a9b-880a-5f13f52def87 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:43.680906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ea9d0c11-958c-4a9b-880a-5f13f52def87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:43.702820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-63c5f267-bbeb-4755-8a50-e1ad510a01be None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:43.708363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-63c5f267-bbeb-4755-8a50-e1ad510a01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:43.835643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 895/1789] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:43.837089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5ed0-400e-4855-8ebb-3074840e9d2f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:43.857272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5ed0-400e-4855-8ebb-3074840e9d2f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:43.857743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5ed0-400e-4855-8ebb-3074840e9d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:43.912713 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-19ab5ed0-400e-4855-8ebb-3074840e9d2f None None] MFA Rules not processed for user `c07403facfaf4c71ab5324af6c44f52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:43.917660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-19ab5ed0-400e-4855-8ebb-3074840e9d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:43.953268 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:44.043449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 896/1790] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 21 05:27:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:44.044201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab5ed0-400e-4855-8ebb-3074840e9d2f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:44.047653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-19ab5ed0-400e-4855-8ebb-3074840e9d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:44.052564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 895/1791] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:44.071954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab5ed0-400e-4855-8ebb-3074840e9d2f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:44.077838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e15898-0c60-4f42-811d-5d9c3510b2bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:44.079131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e15898-0c60-4f42-811d-5d9c3510b2bb service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:44.079421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e15898-0c60-4f42-811d-5d9c3510b2bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:44.079756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e15898-0c60-4f42-811d-5d9c3510b2bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:44.094903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f267-bbeb-4755-8a50-e1ad510a01be None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:44.107996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f267-bbeb-4755-8a50-e1ad510a01be None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:44.126391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20612dbe-8c56-46e9-b63d-14205695b515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:44.127056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20612dbe-8c56-46e9-b63d-14205695b515 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:44.127340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20612dbe-8c56-46e9-b63d-14205695b515 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:44.127614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20612dbe-8c56-46e9-b63d-14205695b515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:44.152451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20612dbe-8c56-46e9-b63d-14205695b515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:44.158746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20612dbe-8c56-46e9-b63d-14205695b515 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:44.175467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e15898-0c60-4f42-811d-5d9c3510b2bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:44.184368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e15898-0c60-4f42-811d-5d9c3510b2bb service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:44.206261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 896/1792] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:44.207181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20612dbe-8c56-46e9-b63d-14205695b515 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:44.216303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 897/1793] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:44.229253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20612dbe-8c56-46e9-b63d-14205695b515 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:44.242870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c0c83-9c23-4e9d-b539-e6d68707ff52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:44.244049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c0c83-9c23-4e9d-b539-e6d68707ff52 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:44.244341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c0c83-9c23-4e9d-b539-e6d68707ff52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:44.244617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c0c83-9c23-4e9d-b539-e6d68707ff52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:44.261407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c0c83-9c23-4e9d-b539-e6d68707ff52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:44.263406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c0c83-9c23-4e9d-b539-e6d68707ff52 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:44.288416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 897/1794] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:44.601347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e15898-0c60-4f42-811d-5d9c3510b2bb service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:44.603766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-07e15898-0c60-4f42-811d-5d9c3510b2bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:44.628124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e15898-0c60-4f42-811d-5d9c3510b2bb service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:44.642901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:44.643593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:44.643888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:44.644374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] PATH_INFO: `/v3/projects/3a75e0c8c75c4b539345332c77539066` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:44.652888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a75e0c8c75c4b539345332c77539066)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:44.659325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:44.802640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a75e0c8c75c4b539345332c77539066', 'request_id': 'req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:44.817884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a75e0c8c75c4b539345332c77539066', 'request_id': 'req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:44.824703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a75e0c8c75c4b539345332c77539066', 'request_id': 'req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:44.852538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] The token cache is being invalidate because project 3a75e0c8c75c4b539345332c77539066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:44.853999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:44.854631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ec2bfc-17e7-409d-b9c8-1397d9a7d956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:44.863334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 898/1795] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:44 2022] DELETE /identity/v3/projects/3a75e0c8c75c4b539345332c77539066 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:44.869485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fb8a-e3c1-460c-8eba-985d3dec913b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:44.869821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fb8a-e3c1-460c-8eba-985d3dec913b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:44.870134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fb8a-e3c1-460c-8eba-985d3dec913b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:44.913947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9517fb8a-e3c1-460c-8eba-985d3dec913b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:44.917110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9517fb8a-e3c1-460c-8eba-985d3dec913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:45.041441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 898/1796] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:45.048981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1531788-e437-4ed9-951b-1c3ddf7f229f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:45.052571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1531788-e437-4ed9-951b-1c3ddf7f229f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:45.053051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1531788-e437-4ed9-951b-1c3ddf7f229f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:45.095682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d1531788-e437-4ed9-951b-1c3ddf7f229f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:45.096999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d1531788-e437-4ed9-951b-1c3ddf7f229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:45.135486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 899/1797] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:45.139662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517fb8a-e3c1-460c-8eba-985d3dec913b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:45.163107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517fb8a-e3c1-460c-8eba-985d3dec913b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:45.186582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ca784-2939-4109-bc63-82e3d373a0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:45.187285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ca784-2939-4109-bc63-82e3d373a0dc admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:45.187571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ca784-2939-4109-bc63-82e3d373a0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:45.187907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ca784-2939-4109-bc63-82e3d373a0dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:45.188772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ca784-2939-4109-bc63-82e3d373a0dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:45.188953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ca784-2939-4109-bc63-82e3d373a0dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:45.190647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ca784-2939-4109-bc63-82e3d373a0dc admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:45.252474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 899/1798] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:45.252679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1531788-e437-4ed9-951b-1c3ddf7f229f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:45.263968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1531788-e437-4ed9-951b-1c3ddf7f229f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:45.285318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7549-f0ec-4c18-af70-954d3df79243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:45.285978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7549-f0ec-4c18-af70-954d3df79243 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:45.286273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7549-f0ec-4c18-af70-954d3df79243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:45.286598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7549-f0ec-4c18-af70-954d3df79243 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:45.287331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee7549-f0ec-4c18-af70-954d3df79243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:45.289276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee7549-f0ec-4c18-af70-954d3df79243 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:45.345848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 900/1799] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:45 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:45.349534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ca784-2939-4109-bc63-82e3d373a0dc admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:45.360015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ca784-2939-4109-bc63-82e3d373a0dc admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:45.492004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:45.508178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06edc5-6ec7-4871-a280-12f5bde2bebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:45.508809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06edc5-6ec7-4871-a280-12f5bde2bebe admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:45.509125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06edc5-6ec7-4871-a280-12f5bde2bebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:45.509403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06edc5-6ec7-4871-a280-12f5bde2bebe admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:45.510075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06edc5-6ec7-4871-a280-12f5bde2bebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:45.515310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06edc5-6ec7-4871-a280-12f5bde2bebe admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:45.614294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 900/1800] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:45 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:45.616660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7549-f0ec-4c18-af70-954d3df79243 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:45.631253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7549-f0ec-4c18-af70-954d3df79243 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:45.646495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c5f05-163c-4357-969b-b97291f17f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:45.647850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c5f05-163c-4357-969b-b97291f17f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:45.647850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c5f05-163c-4357-969b-b97291f17f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:45.647850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c5f05-163c-4357-969b-b97291f17f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:45.648312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c5f05-163c-4357-969b-b97291f17f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:45.650548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c5f05-163c-4357-969b-b97291f17f49 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:45.677072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 901/1801] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:45.681096 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06edc5-6ec7-4871-a280-12f5bde2bebe admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:45.703268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06edc5-6ec7-4871-a280-12f5bde2bebe admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:45.713156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae03841-afc4-4452-bade-68075d6565de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:45.714032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae03841-afc4-4452-bade-68075d6565de admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:45.714319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae03841-afc4-4452-bade-68075d6565de admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:45.714595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae03841-afc4-4452-bade-68075d6565de admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:45.715433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae03841-afc4-4452-bade-68075d6565de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:45.717324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae03841-afc4-4452-bade-68075d6565de admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:45.732765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 901/1802] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:27:45.737585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c5f05-163c-4357-969b-b97291f17f49 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:45.764389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c5f05-163c-4357-969b-b97291f17f49 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:45.779195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab757e-520b-4cc1-a696-09351bd8a3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:45.780500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab757e-520b-4cc1-a696-09351bd8a3fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:45.783524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab757e-520b-4cc1-a696-09351bd8a3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:45.784082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab757e-520b-4cc1-a696-09351bd8a3fe admin admin] PATH_INFO: `/v3/projects/11831e77531547299c6e20c4a490b9da/users/20f4a8f93f0349659ab0231e0ba10616/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:45.838957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab757e-520b-4cc1-a696-09351bd8a3fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=11831e77531547299c6e20c4a490b9da, user_id=20f4a8f93f0349659ab0231e0ba10616, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:45.842586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab757e-520b-4cc1-a696-09351bd8a3fe admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:45.884985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 902/1803] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:45 2022] PUT /identity/v3/projects/11831e77531547299c6e20c4a490b9da/users/20f4a8f93f0349659ab0231e0ba10616/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:45.892037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554f6d0-3698-4876-82ca-4b0ea1ed087c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:45.892349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554f6d0-3698-4876-82ca-4b0ea1ed087c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:45.892597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554f6d0-3698-4876-82ca-4b0ea1ed087c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:45.984645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8554f6d0-3698-4876-82ca-4b0ea1ed087c None None] MFA Rules not processed for user `20f4a8f93f0349659ab0231e0ba10616`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:45.986172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8554f6d0-3698-4876-82ca-4b0ea1ed087c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:46.115882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 902/1804] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:46.117741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab757e-520b-4cc1-a696-09351bd8a3fe admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:46.254215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:46.307505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab757e-520b-4cc1-a696-09351bd8a3fe admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:46.314234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd51c83-fb81-436d-9dba-4e7cde6c90ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:46.314895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd51c83-fb81-436d-9dba-4e7cde6c90ee admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:46.315183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd51c83-fb81-436d-9dba-4e7cde6c90ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:46.315476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd51c83-fb81-436d-9dba-4e7cde6c90ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:46.316562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd51c83-fb81-436d-9dba-4e7cde6c90ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:46.318993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd51c83-fb81-436d-9dba-4e7cde6c90ee admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:46.358217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 903/1805] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:46 2022] POST /identity/v3/projects => generated 385 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:46.362765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554f6d0-3698-4876-82ca-4b0ea1ed087c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:46.399005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554f6d0-3698-4876-82ca-4b0ea1ed087c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:46.524849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad99860-4bbb-4cba-8dc0-11523f8378c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:46.525549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad99860-4bbb-4cba-8dc0-11523f8378c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:46.525835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad99860-4bbb-4cba-8dc0-11523f8378c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:46.526288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad99860-4bbb-4cba-8dc0-11523f8378c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:46.527488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad99860-4bbb-4cba-8dc0-11523f8378c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:46.530537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad99860-4bbb-4cba-8dc0-11523f8378c0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:46.593442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd51c83-fb81-436d-9dba-4e7cde6c90ee admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:46.596097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fcd51c83-fb81-436d-9dba-4e7cde6c90ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:46.604451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 903/1806] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:46 2022] POST /identity/v3/users => generated 299 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:46.615183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad99860-4bbb-4cba-8dc0-11523f8378c0 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:46.626169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd51c83-fb81-436d-9dba-4e7cde6c90ee admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:46.638723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:46.639501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:46.642075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:46.642512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] PATH_INFO: `/v3/projects/d577cf46559f4789b1d96c5eb5c80456` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:46.644055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad99860-4bbb-4cba-8dc0-11523f8378c0 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:46.647204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d577cf46559f4789b1d96c5eb5c80456)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:46.651278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:46.652772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a39937-9420-49da-9b18-77b924d8b2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:46.654239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a39937-9420-49da-9b18-77b924d8b2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:46.655421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a39937-9420-49da-9b18-77b924d8b2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:46.655935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a39937-9420-49da-9b18-77b924d8b2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:46.657736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a39937-9420-49da-9b18-77b924d8b2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:46.660876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a39937-9420-49da-9b18-77b924d8b2ea admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:46.692459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 904/1807] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:46.696310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a39937-9420-49da-9b18-77b924d8b2ea admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:46.728997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a39937-9420-49da-9b18-77b924d8b2ea admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:46.739060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393affa1-b0ee-4fc4-bc78-c289e1da027f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:46.740800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393affa1-b0ee-4fc4-bc78-c289e1da027f admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:46.741203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393affa1-b0ee-4fc4-bc78-c289e1da027f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:46.742094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393affa1-b0ee-4fc4-bc78-c289e1da027f admin admin] PATH_INFO: `/v3/projects/81f3aec0dc4b4e47a9b2f45db3bee300/users/6fd87d006dc74f0f8439ad955e39ffef/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:46.785526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393affa1-b0ee-4fc4-bc78-c289e1da027f admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f3aec0dc4b4e47a9b2f45db3bee300, user_id=6fd87d006dc74f0f8439ad955e39ffef, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:46.788849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393affa1-b0ee-4fc4-bc78-c289e1da027f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:46.824594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd577cf46559f4789b1d96c5eb5c80456', 'request_id': 'req-9cefb681-b42b-47eb-965d-915f7049bffa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:46.824778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 905/1808] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:46 2022] PUT /identity/v3/projects/81f3aec0dc4b4e47a9b2f45db3bee300/users/6fd87d006dc74f0f8439ad955e39ffef/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:46.828630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393affa1-b0ee-4fc4-bc78-c289e1da027f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:46.848309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd577cf46559f4789b1d96c5eb5c80456', 'request_id': 'req-9cefb681-b42b-47eb-965d-915f7049bffa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:46.864991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd577cf46559f4789b1d96c5eb5c80456', 'request_id': 'req-9cefb681-b42b-47eb-965d-915f7049bffa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:46.878346 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:46.896572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] The token cache is being invalidate because project d577cf46559f4789b1d96c5eb5c80456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:46.898546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:46.901646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:46.906575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 904/1809] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:46 2022] DELETE /identity/v3/projects/d577cf46559f4789b1d96c5eb5c80456 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:46.910527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:46.914933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:47.092244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393affa1-b0ee-4fc4-bc78-c289e1da027f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:47.110273 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d985c75-f396-4a0d-ab50-fa2138e6cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:47.114704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d985c75-f396-4a0d-ab50-fa2138e6cfce admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:47.115327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d985c75-f396-4a0d-ab50-fa2138e6cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:47.115807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d985c75-f396-4a0d-ab50-fa2138e6cfce admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:47.119224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d985c75-f396-4a0d-ab50-fa2138e6cfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:47.125402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d985c75-f396-4a0d-ab50-fa2138e6cfce admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:47.188913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 906/1810] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:47.200680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d985c75-f396-4a0d-ab50-fa2138e6cfce admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:47.203450 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1d985c75-f396-4a0d-ab50-fa2138e6cfce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:47.219058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefb681-b42b-47eb-965d-915f7049bffa admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:47.251583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:47.255891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:47.256225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:47.256473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] PATH_INFO: `/v3/users/6d3bfaf67796491ab87643a3f99a1d2a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:47.263667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d3bfaf67796491ab87643a3f99a1d2a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:47.275368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:47.277367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d985c75-f396-4a0d-ab50-fa2138e6cfce admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:47.293325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad71a3-8870-4abc-a3c2-ae99343530ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:47.294354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad71a3-8870-4abc-a3c2-ae99343530ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:47.294869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad71a3-8870-4abc-a3c2-ae99343530ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:47.295349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad71a3-8870-4abc-a3c2-ae99343530ed admin admin] PATH_INFO: `/v3/projects/81f3aec0dc4b4e47a9b2f45db3bee300/users/6fd87d006dc74f0f8439ad955e39ffef/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:47.303355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad71a3-8870-4abc-a3c2-ae99343530ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f3aec0dc4b4e47a9b2f45db3bee300, user_id=6fd87d006dc74f0f8439ad955e39ffef, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:47.306525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad71a3-8870-4abc-a3c2-ae99343530ed admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:47.347347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 907/1811] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:47 2022] PUT /identity/v3/projects/81f3aec0dc4b4e47a9b2f45db3bee300/users/6fd87d006dc74f0f8439ad955e39ffef/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:47.364877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ae908-d9aa-4fef-a298-3087cb114b9a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:47.365921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ae908-d9aa-4fef-a298-3087cb114b9a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:47.382953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ae908-d9aa-4fef-a298-3087cb114b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:47.437728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-266ae908-d9aa-4fef-a298-3087cb114b9a None None] MFA Rules not processed for user `6fd87d006dc74f0f8439ad955e39ffef`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:47.438127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d3bfaf67796491ab87643a3f99a1d2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:47.439009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-266ae908-d9aa-4fef-a298-3087cb114b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:47.441393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:47.459468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d3bfaf67796491ab87643a3f99a1d2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:47.460122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d3bfaf67796491ab87643a3f99a1d2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:47.461258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d3bfaf67796491ab87643a3f99a1d2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:47.481496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d3bfaf67796491ab87643a3f99a1d2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:47.523308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 905/1812] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:46 2022] DELETE /identity/v3/users/6d3bfaf67796491ab87643a3f99a1d2a => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:47.666223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 908/1813] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:47.683450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:47.685392 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:47.692969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ae908-d9aa-4fef-a298-3087cb114b9a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:47.706390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-266ae908-d9aa-4fef-a298-3087cb114b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:47.865875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d53b7-ed03-420e-ba27-9f9e6c7ecb50 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:47.881787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10dc05-6d69-42f9-a5f8-6d820149d002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:47.882568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10dc05-6d69-42f9-a5f8-6d820149d002 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:47.882856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10dc05-6d69-42f9-a5f8-6d820149d002 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:47.883223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10dc05-6d69-42f9-a5f8-6d820149d002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:47.887053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8e10dc05-6d69-42f9-a5f8-6d820149d002 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:47.929233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ae908-d9aa-4fef-a298-3087cb114b9a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:48.022107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10dc05-6d69-42f9-a5f8-6d820149d002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:48.024278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10dc05-6d69-42f9-a5f8-6d820149d002 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:48.038580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:48.054125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 906/1814] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:48.125232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:48.125857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:48.126206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:48.126440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] PATH_INFO: `/v3/projects/5edd4dfbc75d439cad4aaf15ac916464` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:48.133547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5edd4dfbc75d439cad4aaf15ac916464)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:48.136464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:48.241927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5edd4dfbc75d439cad4aaf15ac916464', 'request_id': 'req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.257163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5edd4dfbc75d439cad4aaf15ac916464', 'request_id': 'req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.277578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5edd4dfbc75d439cad4aaf15ac916464', 'request_id': 'req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.296766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] The token cache is being invalidate because project 5edd4dfbc75d439cad4aaf15ac916464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:48.297073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.298943 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e6995b2a-53d0-4fa4-a1c3-fd3d60d86590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.302556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 909/1815] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:47 2022] DELETE /identity/v3/projects/5edd4dfbc75d439cad4aaf15ac916464 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:48.306734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10dc05-6d69-42f9-a5f8-6d820149d002 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:48.331725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10dc05-6d69-42f9-a5f8-6d820149d002 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:48.517287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:48.517994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:48.521286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:48.521645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] PATH_INFO: `/v3/users/54ba427c546b4453b67b742752e11642` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:48.522972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ba427c546b4453b67b742752e11642)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:48.525486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:48.599949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:INSERT=1,keystone:DELETE=6,keystone:SELECT=93 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:48.652616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ba427c546b4453b67b742752e11642'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.673584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ba427c546b4453b67b742752e11642'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.674431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ba427c546b4453b67b742752e11642'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.675061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ba427c546b4453b67b742752e11642'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.694176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88d542e7-1436-4fa9-bf67-c43a7f190027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ba427c546b4453b67b742752e11642'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:48.747179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 907/1816] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:27:48 2022] DELETE /identity/v3/users/54ba427c546b4453b67b742752e11642 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:49.354155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd168a4-d759-47c0-bda0-b82eaf1ae127 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:49.354710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd168a4-d759-47c0-bda0-b82eaf1ae127 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:49.354710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd168a4-d759-47c0-bda0-b82eaf1ae127 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:49.356716 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 910/1817] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:27:49.392273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a944e38-6788-4579-8147-467bd3d0642b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:49.393205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a944e38-6788-4579-8147-467bd3d0642b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:49.393668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a944e38-6788-4579-8147-467bd3d0642b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:49.441389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7a944e38-6788-4579-8147-467bd3d0642b None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:49.443492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7a944e38-6788-4579-8147-467bd3d0642b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:49.643534 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 908/1818] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:27:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:49.799518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd168a4-d759-47c0-bda0-b82eaf1ae127 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:49.801967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-afd168a4-d759-47c0-bda0-b82eaf1ae127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:49.836919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd168a4-d759-47c0-bda0-b82eaf1ae127 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:49.851791 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3956291-d796-4ef8-9e95-732f5ff20c92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:49.852465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3956291-d796-4ef8-9e95-732f5ff20c92 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:49.853308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3956291-d796-4ef8-9e95-732f5ff20c92 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:49.853591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3956291-d796-4ef8-9e95-732f5ff20c92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:49.891307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3956291-d796-4ef8-9e95-732f5ff20c92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:49.893666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3956291-d796-4ef8-9e95-732f5ff20c92 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:49.925262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 911/1819] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:49.936879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a944e38-6788-4579-8147-467bd3d0642b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:49.938324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7a944e38-6788-4579-8147-467bd3d0642b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:50.089732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a944e38-6788-4579-8147-467bd3d0642b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:50.096278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:50.097828 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:50.098459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:50.098935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] PATH_INFO: `/v3/projects/300991d9af2a45a8935d4ba27a111270` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:50.100639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=300991d9af2a45a8935d4ba27a111270)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:50.104975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:50.195771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:50.241733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '300991d9af2a45a8935d4ba27a111270', 'request_id': 'req-3d6de34f-f57b-4488-9557-8e30c153c9a3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.261909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '300991d9af2a45a8935d4ba27a111270', 'request_id': 'req-3d6de34f-f57b-4488-9557-8e30c153c9a3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.281773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '300991d9af2a45a8935d4ba27a111270', 'request_id': 'req-3d6de34f-f57b-4488-9557-8e30c153c9a3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.318637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] The token cache is being invalidate because project 300991d9af2a45a8935d4ba27a111270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:50.318976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.319562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.327485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 909/1820] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:49 2022] DELETE /identity/v3/projects/300991d9af2a45a8935d4ba27a111270 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:50.332446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbf340-e1b9-4243-96fb-42cf7fb65549 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:50.333221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbf340-e1b9-4243-96fb-42cf7fb65549 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:50.333716 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbf340-e1b9-4243-96fb-42cf7fb65549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:50.374909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4abbf340-e1b9-4243-96fb-42cf7fb65549 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:50.376296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4abbf340-e1b9-4243-96fb-42cf7fb65549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:50.392118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:50.394278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:50.476710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 912/1821] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:50.485350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6fbd7-2576-4b1b-b7f0-62ff0fbd8652 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:50.485618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6fbd7-2576-4b1b-b7f0-62ff0fbd8652 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:50.485911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6fbd7-2576-4b1b-b7f0-62ff0fbd8652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:50.506722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6de34f-f57b-4488-9557-8e30c153c9a3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:50.513978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:50.513978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:50.514304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:50.514520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] PATH_INFO: `/v3/projects/aad6b43f24ee4b1a887aaef15b4b0edb` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:50.516702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad6b43f24ee4b1a887aaef15b4b0edb)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:50.519352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:50.547967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d4f6fbd7-2576-4b1b-b7f0-62ff0fbd8652 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:50.549628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d4f6fbd7-2576-4b1b-b7f0-62ff0fbd8652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:50.573436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 913/1822] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:50.575026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6fbd7-2576-4b1b-b7f0-62ff0fbd8652 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:50.587697 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad6b43f24ee4b1a887aaef15b4b0edb', 'request_id': 'req-0909b4e2-9051-4c4d-ad63-ee7bce169383'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.592787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6fbd7-2576-4b1b-b7f0-62ff0fbd8652 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:50.599243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad6b43f24ee4b1a887aaef15b4b0edb', 'request_id': 'req-0909b4e2-9051-4c4d-ad63-ee7bce169383'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.629811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad6b43f24ee4b1a887aaef15b4b0edb', 'request_id': 'req-0909b4e2-9051-4c4d-ad63-ee7bce169383'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.662336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] The token cache is being invalidate because project aad6b43f24ee4b1a887aaef15b4b0edb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:27:50.662582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.665589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0909b4e2-9051-4c4d-ad63-ee7bce169383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:27:50.667595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 910/1823] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:27:50 2022] DELETE /identity/v3/projects/aad6b43f24ee4b1a887aaef15b4b0edb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:50.675591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66386a-cb72-45b5-a0fa-af44f9e440a6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:50.676735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66386a-cb72-45b5-a0fa-af44f9e440a6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:50.677105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66386a-cb72-45b5-a0fa-af44f9e440a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:50.692643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:50.729324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4f66386a-cb72-45b5-a0fa-af44f9e440a6 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:50.730969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:50.731816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:50.732420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:50.732809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4f66386a-cb72-45b5-a0fa-af44f9e440a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:50.733119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:50.733999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:50.734483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:50.736284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:50.745422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 911/1824] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:50.760750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af39ea-b58f-4098-8066-b46ea8ddfa6d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:50.761128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af39ea-b58f-4098-8066-b46ea8ddfa6d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:50.761605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af39ea-b58f-4098-8066-b46ea8ddfa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:50.763197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 914/1825] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:50.770773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:50.772726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:50.791495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da33cd6-af13-4d81-9b78-3982facae43f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:50.798676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69a548-90bf-4c3e-83ca-b670749e0d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:50.799523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69a548-90bf-4c3e-83ca-b670749e0d5e admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:50.800458 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69a548-90bf-4c3e-83ca-b670749e0d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:50.800986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69a548-90bf-4c3e-83ca-b670749e0d5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:50.802117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69a548-90bf-4c3e-83ca-b670749e0d5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:50.805466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69a548-90bf-4c3e-83ca-b670749e0d5e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:50.842922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-14af39ea-b58f-4098-8066-b46ea8ddfa6d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:50.844249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-14af39ea-b58f-4098-8066-b46ea8ddfa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:50.875995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 912/1826] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:50.883356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 915/1827] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:50 2022] POST /identity/v3/projects => generated 357 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:50.887517 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af39ea-b58f-4098-8066-b46ea8ddfa6d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:50.892114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69a548-90bf-4c3e-83ca-b670749e0d5e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:50.919896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af39ea-b58f-4098-8066-b46ea8ddfa6d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:50.989892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:51.003480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69a548-90bf-4c3e-83ca-b670749e0d5e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.020441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed10b9-45b1-4612-be7a-2db58b4f5379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.021065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed10b9-45b1-4612-be7a-2db58b4f5379 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.021353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed10b9-45b1-4612-be7a-2db58b4f5379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.021632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed10b9-45b1-4612-be7a-2db58b4f5379 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.022383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed10b9-45b1-4612-be7a-2db58b4f5379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.024409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed10b9-45b1-4612-be7a-2db58b4f5379 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.066030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 916/1828] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:50 2022] POST /identity/v3/users => generated 279 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:51.067148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed10b9-45b1-4612-be7a-2db58b4f5379 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.073286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787edd0-20ee-4e92-ace4-2c7498b1e0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.073953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787edd0-20ee-4e92-ace4-2c7498b1e0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.074236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787edd0-20ee-4e92-ace4-2c7498b1e0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.074508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787edd0-20ee-4e92-ace4-2c7498b1e0b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.075701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787edd0-20ee-4e92-ace4-2c7498b1e0b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:27:51.075701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787edd0-20ee-4e92-ace4-2c7498b1e0b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.078163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787edd0-20ee-4e92-ace4-2c7498b1e0b9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.096814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed10b9-45b1-4612-be7a-2db58b4f5379 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.109269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db456b82-6ad3-4e0a-8d63-6342ff21e9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.111320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 913/1829] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:27:51.112183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db456b82-6ad3-4e0a-8d63-6342ff21e9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.112593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db456b82-6ad3-4e0a-8d63-6342ff21e9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.112816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db456b82-6ad3-4e0a-8d63-6342ff21e9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.114093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db456b82-6ad3-4e0a-8d63-6342ff21e9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.115800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db456b82-6ad3-4e0a-8d63-6342ff21e9e6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.118902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787edd0-20ee-4e92-ace4-2c7498b1e0b9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.136415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787edd0-20ee-4e92-ace4-2c7498b1e0b9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.139633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 917/1830] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:27:51.145316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db456b82-6ad3-4e0a-8d63-6342ff21e9e6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.155650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f1f13-8707-4850-86e5-900f3823c96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.156378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f1f13-8707-4850-86e5-900f3823c96f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.156667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f1f13-8707-4850-86e5-900f3823c96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.157059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f1f13-8707-4850-86e5-900f3823c96f admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.157691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f1f13-8707-4850-86e5-900f3823c96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.163683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f1f13-8707-4850-86e5-900f3823c96f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.170133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db456b82-6ad3-4e0a-8d63-6342ff21e9e6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.185517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80c198-06dc-4360-9e62-fbd221f557d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.186160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80c198-06dc-4360-9e62-fbd221f557d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.186339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80c198-06dc-4360-9e62-fbd221f557d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.186655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80c198-06dc-4360-9e62-fbd221f557d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.187269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80c198-06dc-4360-9e62-fbd221f557d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.188890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80c198-06dc-4360-9e62-fbd221f557d5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.217811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 918/1831] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:51.218906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80c198-06dc-4360-9e62-fbd221f557d5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.224224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 914/1832] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:51 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:51.229736 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f1f13-8707-4850-86e5-900f3823c96f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.232209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80c198-06dc-4360-9e62-fbd221f557d5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.346644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e34446-50c8-45dc-9f4d-afd72f27d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.347704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e34446-50c8-45dc-9f4d-afd72f27d792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.348129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e34446-50c8-45dc-9f4d-afd72f27d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.348542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e34446-50c8-45dc-9f4d-afd72f27d792 admin admin] PATH_INFO: `/v3/projects/846c6dc450424d788e222b5277848e26/users/194ddf378364483390f434a493315247/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.371717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e34446-50c8-45dc-9f4d-afd72f27d792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846c6dc450424d788e222b5277848e26, user_id=194ddf378364483390f434a493315247, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.375220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e34446-50c8-45dc-9f4d-afd72f27d792 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.397253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f1f13-8707-4850-86e5-900f3823c96f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.400554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 919/1833] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:51 2022] PUT /identity/v3/projects/846c6dc450424d788e222b5277848e26/users/194ddf378364483390f434a493315247/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:51.411637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bad2a5-08e5-4da5-b2b6-1268701f21ac None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.411935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bad2a5-08e5-4da5-b2b6-1268701f21ac None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.412208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bad2a5-08e5-4da5-b2b6-1268701f21ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.424671 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:51.492834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-25bad2a5-08e5-4da5-b2b6-1268701f21ac None None] MFA Rules not processed for user `194ddf378364483390f434a493315247`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:51.494161 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-25bad2a5-08e5-4da5-b2b6-1268701f21ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:51.517226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97305f6d-00e9-42bd-9200-2940162ed10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.517839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97305f6d-00e9-42bd-9200-2940162ed10b admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.518164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97305f6d-00e9-42bd-9200-2940162ed10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.518436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97305f6d-00e9-42bd-9200-2940162ed10b admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.519353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97305f6d-00e9-42bd-9200-2940162ed10b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.521248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97305f6d-00e9-42bd-9200-2940162ed10b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.562635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 915/1834] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:51 2022] POST /identity/v3/users => generated 289 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:51.566455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97305f6d-00e9-42bd-9200-2940162ed10b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.584380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97305f6d-00e9-42bd-9200-2940162ed10b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.602036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ac53d-ab47-49d0-b6f7-0fd904a0b833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.603200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 920/1835] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:51.607336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ac53d-ab47-49d0-b6f7-0fd904a0b833 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.607642 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bad2a5-08e5-4da5-b2b6-1268701f21ac None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.608068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ac53d-ab47-49d0-b6f7-0fd904a0b833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.608561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ac53d-ab47-49d0-b6f7-0fd904a0b833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.609584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ac53d-ab47-49d0-b6f7-0fd904a0b833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.612049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ac53d-ab47-49d0-b6f7-0fd904a0b833 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.625005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 916/1836] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:27:51.629374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ac53d-ab47-49d0-b6f7-0fd904a0b833 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.630611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bad2a5-08e5-4da5-b2b6-1268701f21ac None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.635705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06429f-70fa-4bd5-88eb-5115209988c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.636637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06429f-70fa-4bd5-88eb-5115209988c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.637141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06429f-70fa-4bd5-88eb-5115209988c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.637246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06429f-70fa-4bd5-88eb-5115209988c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.638188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06429f-70fa-4bd5-88eb-5115209988c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.640325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06429f-70fa-4bd5-88eb-5115209988c0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.642278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ac53d-ab47-49d0-b6f7-0fd904a0b833 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.648785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b30386-9950-4ec7-885f-b27674fb9464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.650493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b30386-9950-4ec7-885f-b27674fb9464 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.651079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b30386-9950-4ec7-885f-b27674fb9464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.662705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b30386-9950-4ec7-885f-b27674fb9464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.665029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b30386-9950-4ec7-885f-b27674fb9464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.675972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b30386-9950-4ec7-885f-b27674fb9464 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.681260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 921/1837] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:51 2022] POST /identity/v3/projects => generated 357 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:27:51.686556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb5914-8079-41dc-80eb-701d69e5212a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.686851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb5914-8079-41dc-80eb-701d69e5212a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.687134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb5914-8079-41dc-80eb-701d69e5212a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.688659 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 922/1838] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:27:51.689451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb5914-8079-41dc-80eb-701d69e5212a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.709098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 917/1839] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:27:51.709098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d751d2-9443-41a1-843f-2734b98810ff None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.709098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d751d2-9443-41a1-843f-2734b98810ff None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.709098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d751d2-9443-41a1-843f-2734b98810ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.754941 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-44d751d2-9443-41a1-843f-2734b98810ff None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:51.756542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-44d751d2-9443-41a1-843f-2734b98810ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:51.804771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:51.806877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb5914-8079-41dc-80eb-701d69e5212a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.813515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03939257-adfd-4e28-9900-68dbe2801fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.814172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939257-adfd-4e28-9900-68dbe2801fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.814615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939257-adfd-4e28-9900-68dbe2801fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.815043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939257-adfd-4e28-9900-68dbe2801fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.816557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03939257-adfd-4e28-9900-68dbe2801fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.818869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03939257-adfd-4e28-9900-68dbe2801fc5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.893313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 923/1840] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:51 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:51.894385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03939257-adfd-4e28-9900-68dbe2801fc5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.913335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 918/1841] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:27:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:51.914345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d751d2-9443-41a1-843f-2734b98810ff None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:51.942072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d751d2-9443-41a1-843f-2734b98810ff None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.951840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03939257-adfd-4e28-9900-68dbe2801fc5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:51.959151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786ff83-4d02-4cee-83ff-991568c643f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.961012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786ff83-4d02-4cee-83ff-991568c643f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.961849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786ff83-4d02-4cee-83ff-991568c643f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.962722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786ff83-4d02-4cee-83ff-991568c643f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.963442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786ff83-4d02-4cee-83ff-991568c643f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:51.969955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c40d07-2901-4825-9ec4-8086b6bb48e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:51.970630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c40d07-2901-4825-9ec4-8086b6bb48e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:51.970914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c40d07-2901-4825-9ec4-8086b6bb48e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:51.971186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c40d07-2901-4825-9ec4-8086b6bb48e7 admin admin] PATH_INFO: `/v3/projects/250739247f8948a1b0fb7898f0238a10/users/225183cdc94c473baaf01e1cd8d2f558/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:51.972007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786ff83-4d02-4cee-83ff-991568c643f5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:51.993918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 919/1842] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:51.999158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786ff83-4d02-4cee-83ff-991568c643f5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:52.009929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c40d07-2901-4825-9ec4-8086b6bb48e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250739247f8948a1b0fb7898f0238a10, user_id=225183cdc94c473baaf01e1cd8d2f558, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:52.012886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c40d07-2901-4825-9ec4-8086b6bb48e7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:52.045790 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 924/1843] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:51 2022] PUT /identity/v3/projects/250739247f8948a1b0fb7898f0238a10/users/225183cdc94c473baaf01e1cd8d2f558/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:52.046494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786ff83-4d02-4cee-83ff-991568c643f5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:52.059210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f24350-4b4d-49a8-8c7d-9245e3db2b38 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:52.059659 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f24350-4b4d-49a8-8c7d-9245e3db2b38 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:52.060001 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f24350-4b4d-49a8-8c7d-9245e3db2b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:52.138687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-75f24350-4b4d-49a8-8c7d-9245e3db2b38 None None] MFA Rules not processed for user `225183cdc94c473baaf01e1cd8d2f558`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:52.140005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-75f24350-4b4d-49a8-8c7d-9245e3db2b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:52.193633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:52.211012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddee8e-f3d6-43ec-84bd-796254b81bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:52.212086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddee8e-f3d6-43ec-84bd-796254b81bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:52.212569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddee8e-f3d6-43ec-84bd-796254b81bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:52.212992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddee8e-f3d6-43ec-84bd-796254b81bc1 admin admin] PATH_INFO: `/v3/projects/7484e9774ffc407595d22913b40a2172/users/df6380c9a9b7459083e9e5c4acb178e3/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:52.257981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddee8e-f3d6-43ec-84bd-796254b81bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7484e9774ffc407595d22913b40a2172, user_id=df6380c9a9b7459083e9e5c4acb178e3, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:52.262304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddee8e-f3d6-43ec-84bd-796254b81bc1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:52.311211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 925/1844] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:52.312154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f24350-4b4d-49a8-8c7d-9245e3db2b38 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:52.313664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-75f24350-4b4d-49a8-8c7d-9245e3db2b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:52.319446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 920/1845] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:51 2022] PUT /identity/v3/projects/7484e9774ffc407595d22913b40a2172/users/df6380c9a9b7459083e9e5c4acb178e3/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:52.323971 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddee8e-f3d6-43ec-84bd-796254b81bc1 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:52.452994 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f24350-4b4d-49a8-8c7d-9245e3db2b38 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:52.461627 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddee8e-f3d6-43ec-84bd-796254b81bc1 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:52.474800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373cef9e-8c56-4201-beed-0b58162da806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:52.475438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373cef9e-8c56-4201-beed-0b58162da806 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:52.475732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373cef9e-8c56-4201-beed-0b58162da806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:52.475999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373cef9e-8c56-4201-beed-0b58162da806 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:52.476638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373cef9e-8c56-4201-beed-0b58162da806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:52.489629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373cef9e-8c56-4201-beed-0b58162da806 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:52.555316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 921/1846] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:27:52 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:52.556447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373cef9e-8c56-4201-beed-0b58162da806 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:52.589913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:52.633278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c71a46-324e-4324-81af-d07ce2287771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:52.638289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c71a46-324e-4324-81af-d07ce2287771 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:52.639546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c71a46-324e-4324-81af-d07ce2287771 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:52.640117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c71a46-324e-4324-81af-d07ce2287771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:52.671687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c71a46-324e-4324-81af-d07ce2287771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:52.673376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c71a46-324e-4324-81af-d07ce2287771 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:52.700501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 926/1847] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:52.701535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c71a46-324e-4324-81af-d07ce2287771 service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:52.724170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c71a46-324e-4324-81af-d07ce2287771 service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:52.727336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373cef9e-8c56-4201-beed-0b58162da806 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:52.736796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e3f14-3eea-4ff8-80fe-e0c3ff563f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:52.739668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e56d67-8f32-4c13-be00-815ddf2614a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:52.740085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e3f14-3eea-4ff8-80fe-e0c3ff563f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:52.740444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e3f14-3eea-4ff8-80fe-e0c3ff563f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:52.741263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e56d67-8f32-4c13-be00-815ddf2614a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:52.741455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e3f14-3eea-4ff8-80fe-e0c3ff563f01 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:52.742220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e3f14-3eea-4ff8-80fe-e0c3ff563f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:52.742599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e56d67-8f32-4c13-be00-815ddf2614a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:52.742942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e56d67-8f32-4c13-be00-815ddf2614a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:52.745656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e56d67-8f32-4c13-be00-815ddf2614a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:52.748035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e3f14-3eea-4ff8-80fe-e0c3ff563f01 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:52.749659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e56d67-8f32-4c13-be00-815ddf2614a1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:52.796474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 922/1848] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:52.797187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e56d67-8f32-4c13-be00-815ddf2614a1 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:52.809692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e56d67-8f32-4c13-be00-815ddf2614a1 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:52.815378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ed5cd-57e1-4eb4-afbd-08f27eae478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:52.816069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ed5cd-57e1-4eb4-afbd-08f27eae478a admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:52.816352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ed5cd-57e1-4eb4-afbd-08f27eae478a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:52.816632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ed5cd-57e1-4eb4-afbd-08f27eae478a admin admin] PATH_INFO: `/v3/projects/7484e9774ffc407595d22913b40a2172/users/df6380c9a9b7459083e9e5c4acb178e3/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:52.819076 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ed5cd-57e1-4eb4-afbd-08f27eae478a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7484e9774ffc407595d22913b40a2172, user_id=df6380c9a9b7459083e9e5c4acb178e3, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:52.824028 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ed5cd-57e1-4eb4-afbd-08f27eae478a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:52.837208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 923/1849] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:52 2022] PUT /identity/v3/projects/7484e9774ffc407595d22913b40a2172/users/df6380c9a9b7459083e9e5c4acb178e3/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:52.853181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f7522-2874-4a3b-b449-35092d64bc93 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:52.854174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f7522-2874-4a3b-b449-35092d64bc93 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:52.855949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f7522-2874-4a3b-b449-35092d64bc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:52.862202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 927/1850] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:27:52 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:27:52.863331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e3f14-3eea-4ff8-80fe-e0c3ff563f01 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:52.888358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e3f14-3eea-4ff8-80fe-e0c3ff563f01 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:52.965149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a22f7522-2874-4a3b-b449-35092d64bc93 None None] MFA Rules not processed for user `df6380c9a9b7459083e9e5c4acb178e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:52.967180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a22f7522-2874-4a3b-b449-35092d64bc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:52.970999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:53.041069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0c0e3-83cf-46e9-acde-d8b79370c0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:53.042171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0c0e3-83cf-46e9-acde-d8b79370c0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:53.042960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0c0e3-83cf-46e9-acde-d8b79370c0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:53.042960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0c0e3-83cf-46e9-acde-d8b79370c0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:53.044365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0c0e3-83cf-46e9-acde-d8b79370c0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:53.046553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0c0e3-83cf-46e9-acde-d8b79370c0b2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:53.064883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 928/1851] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:27:53.067268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0c0e3-83cf-46e9-acde-d8b79370c0b2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:53.086268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0c0e3-83cf-46e9-acde-d8b79370c0b2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:53.098168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c022e-d48c-4960-a014-e623b4e85b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:53.098865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c022e-d48c-4960-a014-e623b4e85b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:53.099150 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c022e-d48c-4960-a014-e623b4e85b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:53.099508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c022e-d48c-4960-a014-e623b4e85b4a admin admin] PATH_INFO: `/v3/projects/64a36f687dfe40cb96db29a9ddba51aa/users/56143a362aba44bc8844537100876132/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:53.121262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 924/1852] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:27:53.131194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c022e-d48c-4960-a014-e623b4e85b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a36f687dfe40cb96db29a9ddba51aa, user_id=56143a362aba44bc8844537100876132, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:53.134069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f7522-2874-4a3b-b449-35092d64bc93 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:53.134197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c022e-d48c-4960-a014-e623b4e85b4a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:53.135795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a22f7522-2874-4a3b-b449-35092d64bc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:53.157126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 929/1853] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:53 2022] PUT /identity/v3/projects/64a36f687dfe40cb96db29a9ddba51aa/users/56143a362aba44bc8844537100876132/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:53.161106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c022e-d48c-4960-a014-e623b4e85b4a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:53.265492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f7522-2874-4a3b-b449-35092d64bc93 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:53.334619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:53.344539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c022e-d48c-4960-a014-e623b4e85b4a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:53.359595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a248c98-4bd3-4358-8e6d-cfa4e1244541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:53.360563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a248c98-4bd3-4358-8e6d-cfa4e1244541 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:53.360776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e24500-e6cb-48c6-8ff5-513563613798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:53.361096 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a248c98-4bd3-4358-8e6d-cfa4e1244541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:53.361784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a248c98-4bd3-4358-8e6d-cfa4e1244541 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:53.361894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e24500-e6cb-48c6-8ff5-513563613798 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:53.362290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e24500-e6cb-48c6-8ff5-513563613798 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:53.362705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e24500-e6cb-48c6-8ff5-513563613798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:53.362920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a248c98-4bd3-4358-8e6d-cfa4e1244541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:53.364764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a248c98-4bd3-4358-8e6d-cfa4e1244541 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:53.402358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 930/1854] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:27:53.413097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a248c98-4bd3-4358-8e6d-cfa4e1244541 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:53.444478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a248c98-4bd3-4358-8e6d-cfa4e1244541 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:53.449891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb890840-da5b-4212-8422-c8098c70625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:53.450655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb890840-da5b-4212-8422-c8098c70625e admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:53.451052 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb890840-da5b-4212-8422-c8098c70625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:53.451393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb890840-da5b-4212-8422-c8098c70625e admin admin] PATH_INFO: `/v3/projects/64a36f687dfe40cb96db29a9ddba51aa/users/56143a362aba44bc8844537100876132/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:53.456716 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb890840-da5b-4212-8422-c8098c70625e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a36f687dfe40cb96db29a9ddba51aa, user_id=56143a362aba44bc8844537100876132, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:53.461411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb890840-da5b-4212-8422-c8098c70625e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:53.484390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 931/1855] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:27:53 2022] PUT /identity/v3/projects/64a36f687dfe40cb96db29a9ddba51aa/users/56143a362aba44bc8844537100876132/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:27:53.487512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e24500-e6cb-48c6-8ff5-513563613798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:53.488977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e24500-e6cb-48c6-8ff5-513563613798 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:53.496760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99b751-daae-42c3-b1dd-043d431d0938 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:53.496974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99b751-daae-42c3-b1dd-043d431d0938 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:53.497239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99b751-daae-42c3-b1dd-043d431d0938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:53.503519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 925/1856] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:53 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:27:53.550533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3c99b751-daae-42c3-b1dd-043d431d0938 None None] MFA Rules not processed for user `56143a362aba44bc8844537100876132`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:27:53.551837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3c99b751-daae-42c3-b1dd-043d431d0938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:53.643728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e24500-e6cb-48c6-8ff5-513563613798 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:53.655948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e24500-e6cb-48c6-8ff5-513563613798 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:53.688418 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 932/1857] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:27:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:27:53.701889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99b751-daae-42c3-b1dd-043d431d0938 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:53.703927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3c99b751-daae-42c3-b1dd-043d431d0938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:53.736719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:53.768804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17929499-0f73-4d7e-b731-da329d708ea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:53.770349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17929499-0f73-4d7e-b731-da329d708ea2 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:53.772262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17929499-0f73-4d7e-b731-da329d708ea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:53.772546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17929499-0f73-4d7e-b731-da329d708ea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:53.794044 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99b751-daae-42c3-b1dd-043d431d0938 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:53.797866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bd0d4-a569-441d-b0ec-0c6cee4522d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:53.798514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bd0d4-a569-441d-b0ec-0c6cee4522d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:53.798798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bd0d4-a569-441d-b0ec-0c6cee4522d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:53.799076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bd0d4-a569-441d-b0ec-0c6cee4522d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:53.845843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17929499-0f73-4d7e-b731-da329d708ea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:53.847347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17929499-0f73-4d7e-b731-da329d708ea2 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:53.856632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 926/1858] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:53.859364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bd0d4-a569-441d-b0ec-0c6cee4522d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:53.860817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bd0d4-a569-441d-b0ec-0c6cee4522d6 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:53.870844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 933/1859] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:56.690190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17929499-0f73-4d7e-b731-da329d708ea2 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:56.698656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17929499-0f73-4d7e-b731-da329d708ea2 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:56.704844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:56.705487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:56.705835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:56.706142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:56.708154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:27:56.753649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:27:56.780521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:56.782376 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:56.793135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 927/1860] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:27:59.926903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bd0d4-a569-441d-b0ec-0c6cee4522d6 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:27:59.936080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bd0d4-a569-441d-b0ec-0c6cee4522d6 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:27:59.939719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9d1ef-80df-4f3d-9d21-69e40f95df98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:27:59.940319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9d1ef-80df-4f3d-9d21-69e40f95df98 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:27:59.940599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9d1ef-80df-4f3d-9d21-69e40f95df98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:27:59.940873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9d1ef-80df-4f3d-9d21-69e40f95df98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:27:59.950367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9d1ef-80df-4f3d-9d21-69e40f95df98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:27:59.951820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9d1ef-80df-4f3d-9d21-69e40f95df98 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:27:59.962169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 934/1861] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:27:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:06.788249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=11 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:08.622816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:08.625228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:08.769478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3bcd7-5def-4509-bd98-292603889147 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:08.782910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:08.783571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:08.783859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:08.784127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] PATH_INFO: `/v3/users/20f4a8f93f0349659ab0231e0ba10616` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:08.786365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f4a8f93f0349659ab0231e0ba10616)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:08.788889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:08.878075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f4a8f93f0349659ab0231e0ba10616'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:08.899639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f4a8f93f0349659ab0231e0ba10616'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:08.900387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f4a8f93f0349659ab0231e0ba10616'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:08.901118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f4a8f93f0349659ab0231e0ba10616'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:08.912951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aac1106b-f216-40b3-9385-cb3ac0e6a2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f4a8f93f0349659ab0231e0ba10616'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:08.962369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 928/1862] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:08 2022] DELETE /identity/v3/users/20f4a8f93f0349659ab0231e0ba10616 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:08.967124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca220088-dd66-4b3a-9337-29e3513a0162 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:08.967649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca220088-dd66-4b3a-9337-29e3513a0162 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:08.967740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca220088-dd66-4b3a-9337-29e3513a0162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:09.004783 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ca220088-dd66-4b3a-9337-29e3513a0162 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:09.006275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ca220088-dd66-4b3a-9337-29e3513a0162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:09.082603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8caae2-5393-43cb-97fc-40b45e1eb526 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:09.082916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8caae2-5393-43cb-97fc-40b45e1eb526 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:09.083350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8caae2-5393-43cb-97fc-40b45e1eb526 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:09.084992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 929/1863] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:28:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:28:09.147350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 935/1864] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:09.162933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3d5de-1ab4-4588-866f-fcb400878c3e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:09.163613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3d5de-1ab4-4588-866f-fcb400878c3e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:09.164066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3d5de-1ab4-4588-866f-fcb400878c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:09.166117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca220088-dd66-4b3a-9337-29e3513a0162 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:09.168207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ca220088-dd66-4b3a-9337-29e3513a0162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:09.203710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e6b3d5de-1ab4-4588-866f-fcb400878c3e None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:09.207307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e6b3d5de-1ab4-4588-866f-fcb400878c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:09.319234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca220088-dd66-4b3a-9337-29e3513a0162 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:09.325161 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5531a-d008-47fc-851f-b81dbe0308ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:09.325788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5531a-d008-47fc-851f-b81dbe0308ce service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:09.326875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5531a-d008-47fc-851f-b81dbe0308ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:09.327225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5531a-d008-47fc-851f-b81dbe0308ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:09.339111 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:09.342492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5531a-d008-47fc-851f-b81dbe0308ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:09.345168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5531a-d008-47fc-851f-b81dbe0308ce service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:09.359212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 936/1865] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:09.401320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 930/1866] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:28:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:09.629160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5531a-d008-47fc-851f-b81dbe0308ce service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:09.631085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-75a5531a-d008-47fc-851f-b81dbe0308ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:09.663896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5531a-d008-47fc-851f-b81dbe0308ce service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:09.679610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1999a-aeac-40a5-9c38-faa7b47125b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:09.682115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1999a-aeac-40a5-9c38-faa7b47125b2 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:09.682457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1999a-aeac-40a5-9c38-faa7b47125b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:09.682743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1999a-aeac-40a5-9c38-faa7b47125b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:09.706696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1999a-aeac-40a5-9c38-faa7b47125b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:09.710251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1999a-aeac-40a5-9c38-faa7b47125b2 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:09.739734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 937/1867] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:09.909447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3d5de-1ab4-4588-866f-fcb400878c3e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:09.911987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e6b3d5de-1ab4-4588-866f-fcb400878c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:09.930580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3d5de-1ab4-4588-866f-fcb400878c3e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:09.936789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:09.937447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:09.937727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:09.938036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] PATH_INFO: `/v3/projects/11831e77531547299c6e20c4a490b9da` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:09.939344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11831e77531547299c6e20c4a490b9da)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:09.944116 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:09.958394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:10.006994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11831e77531547299c6e20c4a490b9da', 'request_id': 'req-7f02295d-7485-468a-879b-a408bf401c37'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.017409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11831e77531547299c6e20c4a490b9da', 'request_id': 'req-7f02295d-7485-468a-879b-a408bf401c37'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.024614 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11831e77531547299c6e20c4a490b9da', 'request_id': 'req-7f02295d-7485-468a-879b-a408bf401c37'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.037475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] The token cache is being invalidate because project 11831e77531547299c6e20c4a490b9da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:10.037646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.038477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.040515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 931/1868] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:09 2022] DELETE /identity/v3/projects/11831e77531547299c6e20c4a490b9da => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:10.045529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1999a-aeac-40a5-9c38-faa7b47125b2 service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:10.057363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1999a-aeac-40a5-9c38-faa7b47125b2 service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:10.117327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:10.118778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:10.119063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:10.119338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] PATH_INFO: `/v3/users/6fd87d006dc74f0f8439ad955e39ffef` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:10.120501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd87d006dc74f0f8439ad955e39ffef)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:10.122476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:10.178133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd87d006dc74f0f8439ad955e39ffef'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.178482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd87d006dc74f0f8439ad955e39ffef'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.190680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd87d006dc74f0f8439ad955e39ffef'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.191498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd87d006dc74f0f8439ad955e39ffef'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.200763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98cf7f33-283c-4461-9755-a147be024c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd87d006dc74f0f8439ad955e39ffef'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.233984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 938/1869] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:10 2022] DELETE /identity/v3/users/6fd87d006dc74f0f8439ad955e39ffef => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:10.607136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:10.608549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:10.639874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02295d-7485-468a-879b-a408bf401c37 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:10.724529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:10.725197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:10.725520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:10.725817 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] PATH_INFO: `/v3/projects/81f3aec0dc4b4e47a9b2f45db3bee300` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:10.727858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=81f3aec0dc4b4e47a9b2f45db3bee300)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:10.734372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:10.797662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81f3aec0dc4b4e47a9b2f45db3bee300', 'request_id': 'req-40bb5080-6f77-4895-9bb2-4e763b2532fb'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.812919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81f3aec0dc4b4e47a9b2f45db3bee300', 'request_id': 'req-40bb5080-6f77-4895-9bb2-4e763b2532fb'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.827917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81f3aec0dc4b4e47a9b2f45db3bee300', 'request_id': 'req-40bb5080-6f77-4895-9bb2-4e763b2532fb'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.855903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] The token cache is being invalidate because project 81f3aec0dc4b4e47a9b2f45db3bee300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:10.856763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.858191 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-40bb5080-6f77-4895-9bb2-4e763b2532fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:10.862027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 932/1870] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:10 2022] DELETE /identity/v3/projects/81f3aec0dc4b4e47a9b2f45db3bee300 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:10.868436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb22c8-493a-45b1-b6f2-89c7785c14df None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:10.869106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb22c8-493a-45b1-b6f2-89c7785c14df None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:10.869106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb22c8-493a-45b1-b6f2-89c7785c14df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:10.896786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ebcb22c8-493a-45b1-b6f2-89c7785c14df None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:10.898201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ebcb22c8-493a-45b1-b6f2-89c7785c14df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:10.991691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 939/1871] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:11.000873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a2a15-dc49-4b71-8df1-40ec85608459 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:11.001410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a2a15-dc49-4b71-8df1-40ec85608459 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:11.002201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a2a15-dc49-4b71-8df1-40ec85608459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:11.025240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-324a2a15-dc49-4b71-8df1-40ec85608459 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:11.026745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-324a2a15-dc49-4b71-8df1-40ec85608459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:11.038914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 933/1872] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:11.040778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb22c8-493a-45b1-b6f2-89c7785c14df None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:11.045659 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:11.060619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb22c8-493a-45b1-b6f2-89c7785c14df None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:11.065275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aac9df-8c41-4b04-ad23-2a9c19f31a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:11.066048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aac9df-8c41-4b04-ad23-2a9c19f31a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:11.066333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aac9df-8c41-4b04-ad23-2a9c19f31a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:11.066607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aac9df-8c41-4b04-ad23-2a9c19f31a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:11.067260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aac9df-8c41-4b04-ad23-2a9c19f31a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:11.067612 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aac9df-8c41-4b04-ad23-2a9c19f31a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:11.069162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aac9df-8c41-4b04-ad23-2a9c19f31a99 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:11.081360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 940/1873] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:28:11.085839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a2a15-dc49-4b71-8df1-40ec85608459 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:11.095755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a2a15-dc49-4b71-8df1-40ec85608459 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:11.100362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e795c0-00bf-4c63-8dc8-63fa29a3c282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:11.100965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e795c0-00bf-4c63-8dc8-63fa29a3c282 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:11.101248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e795c0-00bf-4c63-8dc8-63fa29a3c282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:11.101525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e795c0-00bf-4c63-8dc8-63fa29a3c282 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:11.102846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e795c0-00bf-4c63-8dc8-63fa29a3c282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:11.104741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e795c0-00bf-4c63-8dc8-63fa29a3c282 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:11.121297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:11.135183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 934/1874] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:11 2022] POST /identity/v3/projects => generated 361 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:11.138367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aac9df-8c41-4b04-ad23-2a9c19f31a99 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:11.148188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aac9df-8c41-4b04-ad23-2a9c19f31a99 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:11.207247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08c39b-11a5-4e4c-a585-9d4b1f194841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:11.207893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08c39b-11a5-4e4c-a585-9d4b1f194841 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:11.208176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08c39b-11a5-4e4c-a585-9d4b1f194841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:11.208616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08c39b-11a5-4e4c-a585-9d4b1f194841 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:11.209263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08c39b-11a5-4e4c-a585-9d4b1f194841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:11.211264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08c39b-11a5-4e4c-a585-9d4b1f194841 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:11.234460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 941/1875] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:11 2022] POST /identity/v3/users => generated 281 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:11.238772 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e795c0-00bf-4c63-8dc8-63fa29a3c282 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:11.249084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e795c0-00bf-4c63-8dc8-63fa29a3c282 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:11.253642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2e3c8-b517-4dd0-8804-b6f3563a87b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:11.254278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2e3c8-b517-4dd0-8804-b6f3563a87b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:11.254562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2e3c8-b517-4dd0-8804-b6f3563a87b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:11.254839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2e3c8-b517-4dd0-8804-b6f3563a87b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:11.255472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2e3c8-b517-4dd0-8804-b6f3563a87b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:11.257053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2e3c8-b517-4dd0-8804-b6f3563a87b5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:11.267412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 935/1876] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:11.271048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08c39b-11a5-4e4c-a585-9d4b1f194841 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:11.281398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08c39b-11a5-4e4c-a585-9d4b1f194841 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:11.286074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cefde2-e5d6-46dd-b38f-63f2c89ee0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:11.286903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cefde2-e5d6-46dd-b38f-63f2c89ee0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:11.287211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cefde2-e5d6-46dd-b38f-63f2c89ee0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:11.287618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cefde2-e5d6-46dd-b38f-63f2c89ee0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:11.288618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cefde2-e5d6-46dd-b38f-63f2c89ee0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:11.290976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cefde2-e5d6-46dd-b38f-63f2c89ee0c7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:11.302683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 942/1877] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:11.306966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2e3c8-b517-4dd0-8804-b6f3563a87b5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:11.316926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2e3c8-b517-4dd0-8804-b6f3563a87b5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:11.322314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd15bde-4fe8-4354-9b6e-b3006381b8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:11.322982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd15bde-4fe8-4354-9b6e-b3006381b8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:11.323262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd15bde-4fe8-4354-9b6e-b3006381b8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:11.323535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd15bde-4fe8-4354-9b6e-b3006381b8dd admin admin] PATH_INFO: `/v3/projects/35c6a3aec31d419488ba024ea687ff19/users/c349adc1e5054da7b96892d85b9bade7/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:11.339227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd15bde-4fe8-4354-9b6e-b3006381b8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c6a3aec31d419488ba024ea687ff19, user_id=c349adc1e5054da7b96892d85b9bade7, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:11.341854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd15bde-4fe8-4354-9b6e-b3006381b8dd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:11.355123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 936/1878] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:11 2022] PUT /identity/v3/projects/35c6a3aec31d419488ba024ea687ff19/users/c349adc1e5054da7b96892d85b9bade7/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:11.366622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bbb83-4eef-4364-9b82-11f64328c0dc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:11.366906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bbb83-4eef-4364-9b82-11f64328c0dc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:11.367174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bbb83-4eef-4364-9b82-11f64328c0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:11.394633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c48bbb83-4eef-4364-9b82-11f64328c0dc None None] MFA Rules not processed for user `c349adc1e5054da7b96892d85b9bade7`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:11.395854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c48bbb83-4eef-4364-9b82-11f64328c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:11.470470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 943/1879] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:11.482578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd15bde-4fe8-4354-9b6e-b3006381b8dd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:11.484356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fdd15bde-4fe8-4354-9b6e-b3006381b8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:11.540311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd15bde-4fe8-4354-9b6e-b3006381b8dd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:11.543854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdba0c-01f4-4d6f-902d-5e1655de69af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:11.544467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdba0c-01f4-4d6f-902d-5e1655de69af service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:11.544758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdba0c-01f4-4d6f-902d-5e1655de69af service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:11.545031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdba0c-01f4-4d6f-902d-5e1655de69af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:11.553267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdba0c-01f4-4d6f-902d-5e1655de69af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:11.554713 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdba0c-01f4-4d6f-902d-5e1655de69af service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:11.564013 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 937/1880] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:17.964187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bbb83-4eef-4364-9b82-11f64328c0dc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:17.967125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c48bbb83-4eef-4364-9b82-11f64328c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:18.052540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:18.057828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bbb83-4eef-4364-9b82-11f64328c0dc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:18.064025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:18.064705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:18.065222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:18.065496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] PATH_INFO: `/v3/users/194ddf378364483390f434a493315247` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:18.066717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=194ddf378364483390f434a493315247)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:18.069191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:18.133430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '194ddf378364483390f434a493315247'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.134436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '194ddf378364483390f434a493315247'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.143591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '194ddf378364483390f434a493315247'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.144265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '194ddf378364483390f434a493315247'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.155179 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '194ddf378364483390f434a493315247'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.173851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 944/1881] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:17 2022] DELETE /identity/v3/users/194ddf378364483390f434a493315247 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:18.178354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a0d7a-f602-4920-b253-a8169f9c4f3f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:18.178816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a0d7a-f602-4920-b253-a8169f9c4f3f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:18.178915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a0d7a-f602-4920-b253-a8169f9c4f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:18.204951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-009a0d7a-f602-4920-b253-a8169f9c4f3f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:18.206414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-009a0d7a-f602-4920-b253-a8169f9c4f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:18.269822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 938/1882] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:18.280525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:18.281990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:18.341254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eb79f-1262-4f89-8ecd-0b7efff79926 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:18.344886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb6e6e-1220-4054-a514-dc7bebf3cb1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:18.345487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb6e6e-1220-4054-a514-dc7bebf3cb1e service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:18.345766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb6e6e-1220-4054-a514-dc7bebf3cb1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:18.346065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb6e6e-1220-4054-a514-dc7bebf3cb1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:18.355653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb6e6e-1220-4054-a514-dc7bebf3cb1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:18.357072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb6e6e-1220-4054-a514-dc7bebf3cb1e service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:18.366546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 945/1883] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:18.638418 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a0d7a-f602-4920-b253-a8169f9c4f3f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:18.639829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-009a0d7a-f602-4920-b253-a8169f9c4f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:18.653294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a0d7a-f602-4920-b253-a8169f9c4f3f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:18.660399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:18.662257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:18.662674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:18.663081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] PATH_INFO: `/v3/projects/846c6dc450424d788e222b5277848e26` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:18.665084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] RBAC: Authorizing `identity:delete_project(project_id=846c6dc450424d788e222b5277848e26)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:18.667244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:18.750156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846c6dc450424d788e222b5277848e26', 'request_id': 'req-4e16b093-084d-4006-93cf-a90f81d6661d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.758878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846c6dc450424d788e222b5277848e26', 'request_id': 'req-4e16b093-084d-4006-93cf-a90f81d6661d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.763526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:18.774457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846c6dc450424d788e222b5277848e26', 'request_id': 'req-4e16b093-084d-4006-93cf-a90f81d6661d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.786800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] The token cache is being invalidate because project 846c6dc450424d788e222b5277848e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:18.787110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.788075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:18.789965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 939/1884] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:18 2022] DELETE /identity/v3/projects/846c6dc450424d788e222b5277848e26 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:18.793690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb6e6e-1220-4054-a514-dc7bebf3cb1e service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:18.810602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb6e6e-1220-4054-a514-dc7bebf3cb1e service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:18.957602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:18.959438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:18.959732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:18.960013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] PATH_INFO: `/v3/users/df6380c9a9b7459083e9e5c4acb178e3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:18.965777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df6380c9a9b7459083e9e5c4acb178e3)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:18.972146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:19.025338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:19.070318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df6380c9a9b7459083e9e5c4acb178e3'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.071638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df6380c9a9b7459083e9e5c4acb178e3'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.083944 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df6380c9a9b7459083e9e5c4acb178e3'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.086335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df6380c9a9b7459083e9e5c4acb178e3'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.116685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df6380c9a9b7459083e9e5c4acb178e3'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.147848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 946/1885] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:18 2022] DELETE /identity/v3/users/df6380c9a9b7459083e9e5c4acb178e3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:19.338883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:19.342920 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:19.391758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16b093-084d-4006-93cf-a90f81d6661d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:19.496516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:19.496516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:19.498272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:19.498914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] PATH_INFO: `/v3/users/bcab9aa11b2b4fc69412fa749919f2bd` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:19.500664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcab9aa11b2b4fc69412fa749919f2bd)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:19.513174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:19.549426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:19.551100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:19.580912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3673bc-bf45-4ab5-a6c0-cb4ea7c372d4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:19.587641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:19.588343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:19.588630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:19.588903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] PATH_INFO: `/v3/projects/7484e9774ffc407595d22913b40a2172` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:19.590077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7484e9774ffc407595d22913b40a2172)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:19.592064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:19.628686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcab9aa11b2b4fc69412fa749919f2bd'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.639641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcab9aa11b2b4fc69412fa749919f2bd'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.640485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcab9aa11b2b4fc69412fa749919f2bd'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.641917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcab9aa11b2b4fc69412fa749919f2bd'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.654408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21e25acb-d708-4a22-a939-a43d41bcacf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcab9aa11b2b4fc69412fa749919f2bd'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.668889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7484e9774ffc407595d22913b40a2172', 'request_id': 'req-54443a79-4db7-403a-b1a9-8c49b6d6bef1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.680583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7484e9774ffc407595d22913b40a2172', 'request_id': 'req-54443a79-4db7-403a-b1a9-8c49b6d6bef1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.684862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 940/1886] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:19 2022] DELETE /identity/v3/users/bcab9aa11b2b4fc69412fa749919f2bd => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:19.686850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67381944-d9d4-4e3c-8f63-b52af0abf025 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:19.687228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67381944-d9d4-4e3c-8f63-b52af0abf025 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:19.687649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67381944-d9d4-4e3c-8f63-b52af0abf025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:19.692471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7484e9774ffc407595d22913b40a2172', 'request_id': 'req-54443a79-4db7-403a-b1a9-8c49b6d6bef1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.710950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] The token cache is being invalidate because project 7484e9774ffc407595d22913b40a2172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:19.711521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.712361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54443a79-4db7-403a-b1a9-8c49b6d6bef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:19.714541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-67381944-d9d4-4e3c-8f63-b52af0abf025 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:19.716375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-67381944-d9d4-4e3c-8f63-b52af0abf025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:19.718101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 947/1887] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:19 2022] DELETE /identity/v3/projects/7484e9774ffc407595d22913b40a2172 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:19.719577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896f429-ef28-466f-83be-8690aee64dd4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:19.722305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896f429-ef28-466f-83be-8690aee64dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:19.722745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896f429-ef28-466f-83be-8690aee64dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:19.752392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d896f429-ef28-466f-83be-8690aee64dd4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:19.758548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d896f429-ef28-466f-83be-8690aee64dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:19.806526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 941/1888] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:19.818428 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67381944-d9d4-4e3c-8f63-b52af0abf025 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:19.821310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-67381944-d9d4-4e3c-8f63-b52af0abf025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:19.835040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:19.842647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 948/1889] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:19.851082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7890d9-00da-43b0-9300-f0954c628827 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:19.851222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7890d9-00da-43b0-9300-f0954c628827 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:19.851497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7890d9-00da-43b0-9300-f0954c628827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:19.875724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8b7890d9-00da-43b0-9300-f0954c628827 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:19.877201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8b7890d9-00da-43b0-9300-f0954c628827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:19.887888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 949/1890] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:19.892861 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7890d9-00da-43b0-9300-f0954c628827 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:19.906413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7890d9-00da-43b0-9300-f0954c628827 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:19.909891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67381944-d9d4-4e3c-8f63-b52af0abf025 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:19.913843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187ff8e-5500-4482-9312-e154ff9eba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:19.915207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187ff8e-5500-4482-9312-e154ff9eba66 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:19.915559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187ff8e-5500-4482-9312-e154ff9eba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:19.915901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187ff8e-5500-4482-9312-e154ff9eba66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:19.916725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187ff8e-5500-4482-9312-e154ff9eba66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:19.917048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187ff8e-5500-4482-9312-e154ff9eba66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:19.919956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bc662-1aa6-4fa3-ae0b-6e0b2bc25b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:19.920097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187ff8e-5500-4482-9312-e154ff9eba66 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:19.920175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bc662-1aa6-4fa3-ae0b-6e0b2bc25b15 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:19.920175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bc662-1aa6-4fa3-ae0b-6e0b2bc25b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:19.920175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bc662-1aa6-4fa3-ae0b-6e0b2bc25b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:19.934397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bc662-1aa6-4fa3-ae0b-6e0b2bc25b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:19.935984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bc662-1aa6-4fa3-ae0b-6e0b2bc25b15 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:19.941350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 950/1891] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:28:19.945193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187ff8e-5500-4482-9312-e154ff9eba66 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:19.951660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:19.957746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187ff8e-5500-4482-9312-e154ff9eba66 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:19.966384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bffd80-c7b5-4a8d-9f2e-27792ee8bed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:19.967174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bffd80-c7b5-4a8d-9f2e-27792ee8bed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:19.967681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bffd80-c7b5-4a8d-9f2e-27792ee8bed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:19.968054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bffd80-c7b5-4a8d-9f2e-27792ee8bed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:19.969028 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bffd80-c7b5-4a8d-9f2e-27792ee8bed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:19.972342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bffd80-c7b5-4a8d-9f2e-27792ee8bed4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:19.986753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 942/1892] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:20.030497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 951/1893] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:19 2022] POST /identity/v3/projects => generated 401 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:20.034720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bc662-1aa6-4fa3-ae0b-6e0b2bc25b15 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.050508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bc662-1aa6-4fa3-ae0b-6e0b2bc25b15 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.133864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d791281-2516-4a48-89b5-99f889016c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.134569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d791281-2516-4a48-89b5-99f889016c5f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.138291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d791281-2516-4a48-89b5-99f889016c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.138715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d791281-2516-4a48-89b5-99f889016c5f admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.139515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d791281-2516-4a48-89b5-99f889016c5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.141459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d791281-2516-4a48-89b5-99f889016c5f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.170932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 943/1894] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:20 2022] POST /identity/v3/users => generated 301 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:20.171187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bffd80-c7b5-4a8d-9f2e-27792ee8bed4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.188647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bffd80-c7b5-4a8d-9f2e-27792ee8bed4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.195227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082554e-f982-432d-8d3a-8fe621798ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.196718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082554e-f982-432d-8d3a-8fe621798ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.197066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082554e-f982-432d-8d3a-8fe621798ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.197414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082554e-f982-432d-8d3a-8fe621798ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.198186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082554e-f982-432d-8d3a-8fe621798ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.199881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082554e-f982-432d-8d3a-8fe621798ed5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.224139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 952/1895] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:20.225459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d791281-2516-4a48-89b5-99f889016c5f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.236963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d791281-2516-4a48-89b5-99f889016c5f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.242016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106e198-07e3-49cb-bea0-c72e774fcdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.243104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106e198-07e3-49cb-bea0-c72e774fcdff admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.243494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106e198-07e3-49cb-bea0-c72e774fcdff admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.243840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106e198-07e3-49cb-bea0-c72e774fcdff admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.244803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106e198-07e3-49cb-bea0-c72e774fcdff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.261258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106e198-07e3-49cb-bea0-c72e774fcdff admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.285625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 944/1896] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:28:20.286962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082554e-f982-432d-8d3a-8fe621798ed5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.300279 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082554e-f982-432d-8d3a-8fe621798ed5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.315642 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0946dae-23ae-491e-8a76-c38f81eb9dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.316595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0946dae-23ae-491e-8a76-c38f81eb9dda admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.316945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0946dae-23ae-491e-8a76-c38f81eb9dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.317289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0946dae-23ae-491e-8a76-c38f81eb9dda admin admin] PATH_INFO: `/v3/projects/8446d8180dbf4b438d5b26af80ea9a8d/users/9941c3f9d9a04169b00a8f5c03d291b3/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.338082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0946dae-23ae-491e-8a76-c38f81eb9dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=8446d8180dbf4b438d5b26af80ea9a8d, user_id=9941c3f9d9a04169b00a8f5c03d291b3, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.341517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0946dae-23ae-491e-8a76-c38f81eb9dda admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.362106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 953/1897] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:20 2022] PUT /identity/v3/projects/8446d8180dbf4b438d5b26af80ea9a8d/users/9941c3f9d9a04169b00a8f5c03d291b3/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:20.372543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f892a-5b71-449d-a69e-efb1183491d6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.373058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f892a-5b71-449d-a69e-efb1183491d6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.373168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f892a-5b71-449d-a69e-efb1183491d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.405884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d08f892a-5b71-449d-a69e-efb1183491d6 None None] MFA Rules not processed for user `9941c3f9d9a04169b00a8f5c03d291b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:20.407233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d08f892a-5b71-449d-a69e-efb1183491d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:20.493873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 945/1898] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:20 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:20.495118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0946dae-23ae-491e-8a76-c38f81eb9dda admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.579850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0946dae-23ae-491e-8a76-c38f81eb9dda admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.584504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9f80e-7049-4b71-8415-1b120e8447b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.585120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9f80e-7049-4b71-8415-1b120e8447b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.585997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9f80e-7049-4b71-8415-1b120e8447b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.586322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9f80e-7049-4b71-8415-1b120e8447b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.586968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9f80e-7049-4b71-8415-1b120e8447b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.588871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9f80e-7049-4b71-8415-1b120e8447b2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.619724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 954/1899] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:20 2022] POST /identity/v3/projects => generated 399 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:20.623716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f892a-5b71-449d-a69e-efb1183491d6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.635738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f892a-5b71-449d-a69e-efb1183491d6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.649231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:20.702168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3f61-e28d-4c7c-93e5-462a264ca896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.702788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3f61-e28d-4c7c-93e5-462a264ca896 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.703074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3f61-e28d-4c7c-93e5-462a264ca896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.703420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3f61-e28d-4c7c-93e5-462a264ca896 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.704074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f3f61-e28d-4c7c-93e5-462a264ca896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.706050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f3f61-e28d-4c7c-93e5-462a264ca896 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.733255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 946/1900] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:20 2022] POST /identity/v3/users => generated 306 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:20.737347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9f80e-7049-4b71-8415-1b120e8447b2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.748862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9f80e-7049-4b71-8415-1b120e8447b2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.754246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d1d85-c3e4-4386-be7d-fd5e861d2765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.754997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d1d85-c3e4-4386-be7d-fd5e861d2765 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.755382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d1d85-c3e4-4386-be7d-fd5e861d2765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.756282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d1d85-c3e4-4386-be7d-fd5e861d2765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.756516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d1d85-c3e4-4386-be7d-fd5e861d2765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.761056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d1d85-c3e4-4386-be7d-fd5e861d2765 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.782530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 955/1901] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:20.786694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3f61-e28d-4c7c-93e5-462a264ca896 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.797826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3f61-e28d-4c7c-93e5-462a264ca896 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.803787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602d213-59fc-41b7-9dfc-05a9827b5420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.804520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602d213-59fc-41b7-9dfc-05a9827b5420 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.805436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602d213-59fc-41b7-9dfc-05a9827b5420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.805769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602d213-59fc-41b7-9dfc-05a9827b5420 admin admin] PATH_INFO: `/v3/projects/0a4a7b485c614c779bd062be90e74a9c/users/2ee8c9d453664c5cb9eb1734875856b5/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.819334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d1d85-c3e4-4386-be7d-fd5e861d2765 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.822849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4e7d1d85-c3e4-4386-be7d-fd5e861d2765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:20.823995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602d213-59fc-41b7-9dfc-05a9827b5420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4a7b485c614c779bd062be90e74a9c, user_id=2ee8c9d453664c5cb9eb1734875856b5, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.826851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602d213-59fc-41b7-9dfc-05a9827b5420 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.846506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d1d85-c3e4-4386-be7d-fd5e861d2765 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.848524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 947/1902] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:20 2022] PUT /identity/v3/projects/0a4a7b485c614c779bd062be90e74a9c/users/2ee8c9d453664c5cb9eb1734875856b5/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:20.853646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602d213-59fc-41b7-9dfc-05a9827b5420 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.933580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.934908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.935292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.935777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] PATH_INFO: `/v3/projects/474d54a1827748d6a21422ace20a70c9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.937671 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=474d54a1827748d6a21422ace20a70c9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.940352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.942056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602d213-59fc-41b7-9dfc-05a9827b5420 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.950989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694bc07-4b03-457b-b2e9-e64e395e07bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.951730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694bc07-4b03-457b-b2e9-e64e395e07bc admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.952085 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694bc07-4b03-457b-b2e9-e64e395e07bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.952433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694bc07-4b03-457b-b2e9-e64e395e07bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.953155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694bc07-4b03-457b-b2e9-e64e395e07bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.954865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694bc07-4b03-457b-b2e9-e64e395e07bc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:20.965851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 948/1903] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:20.969761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694bc07-4b03-457b-b2e9-e64e395e07bc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:20.975145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:20.985229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694bc07-4b03-457b-b2e9-e64e395e07bc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:20.991192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d0198-c4c4-4229-9dab-a6d69149daa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:20.992059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0198-c4c4-4229-9dab-a6d69149daa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:20.993062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0198-c4c4-4229-9dab-a6d69149daa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:20.993437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0198-c4c4-4229-9dab-a6d69149daa5 admin admin] PATH_INFO: `/v3/projects/0a4a7b485c614c779bd062be90e74a9c/users/2ee8c9d453664c5cb9eb1734875856b5/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:20.995710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d0198-c4c4-4229-9dab-a6d69149daa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4a7b485c614c779bd062be90e74a9c, user_id=2ee8c9d453664c5cb9eb1734875856b5, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:20.999412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d0198-c4c4-4229-9dab-a6d69149daa5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:21.012947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 949/1904] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:20 2022] PUT /identity/v3/projects/0a4a7b485c614c779bd062be90e74a9c/users/2ee8c9d453664c5cb9eb1734875856b5/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:21.024990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ff666-6b88-4355-8f77-a7f64d009736 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:21.026469 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ff666-6b88-4355-8f77-a7f64d009736 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:21.026858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ff666-6b88-4355-8f77-a7f64d009736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:21.041431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474d54a1827748d6a21422ace20a70c9', 'request_id': 'req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.053298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474d54a1827748d6a21422ace20a70c9', 'request_id': 'req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.064676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474d54a1827748d6a21422ace20a70c9', 'request_id': 'req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.080577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] The token cache is being invalidate because project 474d54a1827748d6a21422ace20a70c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:21.081160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.082310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.084340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 956/1905] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:20 2022] DELETE /identity/v3/projects/474d54a1827748d6a21422ace20a70c9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:21.088325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:21.090018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:21.098927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0a9ff666-6b88-4355-8f77-a7f64d009736 None None] MFA Rules not processed for user `2ee8c9d453664c5cb9eb1734875856b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:21.100143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0a9ff666-6b88-4355-8f77-a7f64d009736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:21.163084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:21.180675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3a525-0db8-4e05-a5f1-8c139da9a6a6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:21.189288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 950/1906] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:21 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:21.201242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ff666-6b88-4355-8f77-a7f64d009736 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:21.203141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0a9ff666-6b88-4355-8f77-a7f64d009736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:21.266996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:21.267812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:21.268099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:21.268381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] PATH_INFO: `/v3/users/f47bd7c7f4854eb28623af74a386c2a2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:21.269672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f47bd7c7f4854eb28623af74a386c2a2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:21.271821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:21.311027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ff666-6b88-4355-8f77-a7f64d009736 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:21.316238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ec42d-9e27-48b8-9dca-d08c9dbd5780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:21.317322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ec42d-9e27-48b8-9dca-d08c9dbd5780 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:21.317706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ec42d-9e27-48b8-9dca-d08c9dbd5780 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:21.318172 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ec42d-9e27-48b8-9dca-d08c9dbd5780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:21.331747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ec42d-9e27-48b8-9dca-d08c9dbd5780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:21.334667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ec42d-9e27-48b8-9dca-d08c9dbd5780 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:21.349433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f47bd7c7f4854eb28623af74a386c2a2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.350172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f47bd7c7f4854eb28623af74a386c2a2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.364134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f47bd7c7f4854eb28623af74a386c2a2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.364857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f47bd7c7f4854eb28623af74a386c2a2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.372638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 951/1907] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:21 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:21.375278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:21.379027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f47bd7c7f4854eb28623af74a386c2a2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:21.415588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 957/1908] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:21 2022] DELETE /identity/v3/users/f47bd7c7f4854eb28623af74a386c2a2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:21.511660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ec42d-9e27-48b8-9dca-d08c9dbd5780 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:21.513333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1c9ec42d-9e27-48b8-9dca-d08c9dbd5780 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:21.535951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ec42d-9e27-48b8-9dca-d08c9dbd5780 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:21.631741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d1ebf-d882-49b6-94a9-93e6e6cc83d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:21.632439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d1ebf-d882-49b6-94a9-93e6e6cc83d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:21.632956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d1ebf-d882-49b6-94a9-93e6e6cc83d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:21.633300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d1ebf-d882-49b6-94a9-93e6e6cc83d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:21.636187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-271d1ebf-d882-49b6-94a9-93e6e6cc83d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:21.737918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d1ebf-d882-49b6-94a9-93e6e6cc83d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:21.739650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d1ebf-d882-49b6-94a9-93e6e6cc83d9 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:21.752066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 952/1909] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:21 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:21.835644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:21.837724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:21.970680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8d25b-ea8a-412b-908c-682d3a799787 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:21.977020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:21.978566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:21.979047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:21.979362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] PATH_INFO: `/v3/projects/100c3670ae954300bded6c315d9c9b18` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:21.980783 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=100c3670ae954300bded6c315d9c9b18)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:21.982928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:22.050585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100c3670ae954300bded6c315d9c9b18', 'request_id': 'req-1e730d68-41cb-4cdb-9557-72df2701ea5c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:22.061083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100c3670ae954300bded6c315d9c9b18', 'request_id': 'req-1e730d68-41cb-4cdb-9557-72df2701ea5c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:22.077370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100c3670ae954300bded6c315d9c9b18', 'request_id': 'req-1e730d68-41cb-4cdb-9557-72df2701ea5c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:22.112007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] The token cache is being invalidate because project 100c3670ae954300bded6c315d9c9b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:22.112308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:22.114615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1e730d68-41cb-4cdb-9557-72df2701ea5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:22.120274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 958/1910] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:21 2022] DELETE /identity/v3/projects/100c3670ae954300bded6c315d9c9b18 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:22.121968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24504f99-bbe8-47fc-b838-085170533a77 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:22.122284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24504f99-bbe8-47fc-b838-085170533a77 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:22.122612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24504f99-bbe8-47fc-b838-085170533a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:22.156687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-24504f99-bbe8-47fc-b838-085170533a77 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:22.158215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-24504f99-bbe8-47fc-b838-085170533a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:22.263926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:22.299307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43102828-840a-4684-a742-661763422638 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:22.299732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43102828-840a-4684-a742-661763422638 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:22.300072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43102828-840a-4684-a742-661763422638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:22.314031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 953/1911] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:22.330237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-43102828-840a-4684-a742-661763422638 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:22.331485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-43102828-840a-4684-a742-661763422638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:22.344297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 959/1912] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:22.345162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24504f99-bbe8-47fc-b838-085170533a77 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:22.365743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24504f99-bbe8-47fc-b838-085170533a77 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:22.372528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc39dd3-fd07-4bb6-8b00-884708b63a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:22.373161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc39dd3-fd07-4bb6-8b00-884708b63a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:22.373730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc39dd3-fd07-4bb6-8b00-884708b63a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:22.373730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc39dd3-fd07-4bb6-8b00-884708b63a6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:22.374441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc39dd3-fd07-4bb6-8b00-884708b63a6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:22.374746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc39dd3-fd07-4bb6-8b00-884708b63a6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:22.376893 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc39dd3-fd07-4bb6-8b00-884708b63a6a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:22.395470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 954/1913] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:28:22.397316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43102828-840a-4684-a742-661763422638 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:22.407828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43102828-840a-4684-a742-661763422638 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:22.425379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aff083-effe-473c-969a-162f6e6e4a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:22.426678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aff083-effe-473c-969a-162f6e6e4a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:22.426678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aff083-effe-473c-969a-162f6e6e4a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:22.427269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aff083-effe-473c-969a-162f6e6e4a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:22.428365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aff083-effe-473c-969a-162f6e6e4a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:22.431769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aff083-effe-473c-969a-162f6e6e4a04 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:22.469992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 960/1914] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:22 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:22.471752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc39dd3-fd07-4bb6-8b00-884708b63a6a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:22.483232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc39dd3-fd07-4bb6-8b00-884708b63a6a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:22.563540 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0771c47-967f-40eb-8fde-4e96ca11f8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:22.564877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0771c47-967f-40eb-8fde-4e96ca11f8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:22.564877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0771c47-967f-40eb-8fde-4e96ca11f8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:22.564877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0771c47-967f-40eb-8fde-4e96ca11f8b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:22.565674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0771c47-967f-40eb-8fde-4e96ca11f8b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:22.568080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0771c47-967f-40eb-8fde-4e96ca11f8b4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:22.628354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 955/1915] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:22 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:22.630202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aff083-effe-473c-969a-162f6e6e4a04 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:22.653454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aff083-effe-473c-969a-162f6e6e4a04 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:22.653454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ef72c-662b-43db-939b-1223dc0b179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:22.654539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ef72c-662b-43db-939b-1223dc0b179f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:22.655511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ef72c-662b-43db-939b-1223dc0b179f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:22.655865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ef72c-662b-43db-939b-1223dc0b179f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:22.656604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ef72c-662b-43db-939b-1223dc0b179f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:22.658356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ef72c-662b-43db-939b-1223dc0b179f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:22.673535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 961/1916] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:22.675222 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0771c47-967f-40eb-8fde-4e96ca11f8b4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:22.687240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0771c47-967f-40eb-8fde-4e96ca11f8b4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:22.692261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9da875-f2e6-4539-bc53-f7eb6f8174ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:22.692959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9da875-f2e6-4539-bc53-f7eb6f8174ee admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:22.693309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9da875-f2e6-4539-bc53-f7eb6f8174ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:22.693644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9da875-f2e6-4539-bc53-f7eb6f8174ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:22.694405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9da875-f2e6-4539-bc53-f7eb6f8174ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:22.696159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9da875-f2e6-4539-bc53-f7eb6f8174ee admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:22.714023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 956/1917] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:22.715592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ef72c-662b-43db-939b-1223dc0b179f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:22.729727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ef72c-662b-43db-939b-1223dc0b179f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:22.736909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86849374-08e1-4479-9105-a9a6cc3e8cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:22.737427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86849374-08e1-4479-9105-a9a6cc3e8cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:22.737806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86849374-08e1-4479-9105-a9a6cc3e8cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:22.738216 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86849374-08e1-4479-9105-a9a6cc3e8cc5 admin admin] PATH_INFO: `/v3/projects/196a0dec4e2a4cdeb67d3b7496a5d982/users/c93a57f357554554ac5d92c1b1d836ec/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:22.757352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86849374-08e1-4479-9105-a9a6cc3e8cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=196a0dec4e2a4cdeb67d3b7496a5d982, user_id=c93a57f357554554ac5d92c1b1d836ec, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:22.760743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86849374-08e1-4479-9105-a9a6cc3e8cc5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:22.778136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 962/1918] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:22 2022] PUT /identity/v3/projects/196a0dec4e2a4cdeb67d3b7496a5d982/users/c93a57f357554554ac5d92c1b1d836ec/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:22.791092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573a6a6-15d9-4235-a96b-f5c300049504 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:22.791433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573a6a6-15d9-4235-a96b-f5c300049504 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:22.791838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573a6a6-15d9-4235-a96b-f5c300049504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:22.823364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d573a6a6-15d9-4235-a96b-f5c300049504 None None] MFA Rules not processed for user `c93a57f357554554ac5d92c1b1d836ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:22.824933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d573a6a6-15d9-4235-a96b-f5c300049504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:22.919119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 957/1919] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:22.922536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86849374-08e1-4479-9105-a9a6cc3e8cc5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:22.929550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:23.035394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86849374-08e1-4479-9105-a9a6cc3e8cc5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:23.044758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4f847-befa-4e07-b711-d8b474bf4829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:23.045492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4f847-befa-4e07-b711-d8b474bf4829 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:23.045831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4f847-befa-4e07-b711-d8b474bf4829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:23.046278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4f847-befa-4e07-b711-d8b474bf4829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:23.047349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4f847-befa-4e07-b711-d8b474bf4829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:23.050003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4f847-befa-4e07-b711-d8b474bf4829 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:23.081918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 963/1920] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:22 2022] POST /identity/v3/projects => generated 365 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:23.085754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573a6a6-15d9-4235-a96b-f5c300049504 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:23.098672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573a6a6-15d9-4235-a96b-f5c300049504 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:23.146225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:23.165251 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed5134-5419-4cd6-ac42-abb18c0320f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:23.165878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed5134-5419-4cd6-ac42-abb18c0320f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:23.166201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed5134-5419-4cd6-ac42-abb18c0320f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:23.166478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed5134-5419-4cd6-ac42-abb18c0320f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:23.167120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed5134-5419-4cd6-ac42-abb18c0320f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:23.169053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed5134-5419-4cd6-ac42-abb18c0320f7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:23.192380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 958/1921] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:23 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:23.197184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4f847-befa-4e07-b711-d8b474bf4829 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:23.207473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4f847-befa-4e07-b711-d8b474bf4829 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:23.214851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2de2c-2715-4900-8c3a-b7d3a9270823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:23.215629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2de2c-2715-4900-8c3a-b7d3a9270823 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:23.215926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2de2c-2715-4900-8c3a-b7d3a9270823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:23.216182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2de2c-2715-4900-8c3a-b7d3a9270823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:23.216989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2de2c-2715-4900-8c3a-b7d3a9270823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:23.218839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2de2c-2715-4900-8c3a-b7d3a9270823 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:23.232393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 964/1922] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:23.236673 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed5134-5419-4cd6-ac42-abb18c0320f7 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:23.258914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed5134-5419-4cd6-ac42-abb18c0320f7 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:23.271019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b1124-174c-435e-a850-79088017b625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:23.271872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b1124-174c-435e-a850-79088017b625 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:23.272162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b1124-174c-435e-a850-79088017b625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:23.272485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b1124-174c-435e-a850-79088017b625 admin admin] PATH_INFO: `/v3/projects/da9afbf3d2a54593bb1daf3d0237a44c/users/c74a451b0043491996c5f523d1afab22/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:23.314247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b1124-174c-435e-a850-79088017b625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9afbf3d2a54593bb1daf3d0237a44c, user_id=c74a451b0043491996c5f523d1afab22, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:23.317458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b1124-174c-435e-a850-79088017b625 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:23.356601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 959/1923] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:23 2022] PUT /identity/v3/projects/da9afbf3d2a54593bb1daf3d0237a44c/users/c74a451b0043491996c5f523d1afab22/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:23.360655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2de2c-2715-4900-8c3a-b7d3a9270823 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:23.381003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2de2c-2715-4900-8c3a-b7d3a9270823 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:23.603984 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373eea41-a453-411d-954a-037ccc12a7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:23.604614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eea41-a453-411d-954a-037ccc12a7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:23.604896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eea41-a453-411d-954a-037ccc12a7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:23.605179 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eea41-a453-411d-954a-037ccc12a7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:23.605820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eea41-a453-411d-954a-037ccc12a7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:23.620924 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eea41-a453-411d-954a-037ccc12a7c8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:23.649572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 965/1924] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:23.653629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b1124-174c-435e-a850-79088017b625 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:23.674899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b1124-174c-435e-a850-79088017b625 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:23.689077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db735f64-e20b-4f2c-8742-41b5340561f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:23.689077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db735f64-e20b-4f2c-8742-41b5340561f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:23.689077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db735f64-e20b-4f2c-8742-41b5340561f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:23.689077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db735f64-e20b-4f2c-8742-41b5340561f6 admin admin] PATH_INFO: `/v3/projects/da9afbf3d2a54593bb1daf3d0237a44c/users/c74a451b0043491996c5f523d1afab22/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:23.690869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db735f64-e20b-4f2c-8742-41b5340561f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9afbf3d2a54593bb1daf3d0237a44c, user_id=c74a451b0043491996c5f523d1afab22, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:23.693561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db735f64-e20b-4f2c-8742-41b5340561f6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:23.726979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 960/1925] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:23 2022] PUT /identity/v3/projects/da9afbf3d2a54593bb1daf3d0237a44c/users/c74a451b0043491996c5f523d1afab22/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:23.740971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcff4af-1cc6-4e5a-a04b-b25e85f9095e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:23.741246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcff4af-1cc6-4e5a-a04b-b25e85f9095e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:23.741546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcff4af-1cc6-4e5a-a04b-b25e85f9095e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:23.800327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5bcff4af-1cc6-4e5a-a04b-b25e85f9095e None None] MFA Rules not processed for user `c74a451b0043491996c5f523d1afab22`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:23.801552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5bcff4af-1cc6-4e5a-a04b-b25e85f9095e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:23.981148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:23.996218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 966/1926] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:24.005065 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db735f64-e20b-4f2c-8742-41b5340561f6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:24.008724 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-db735f64-e20b-4f2c-8742-41b5340561f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:24.084119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db735f64-e20b-4f2c-8742-41b5340561f6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:24.088929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8c3cf-a04c-4734-85f5-62ef440dd440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:24.089543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8c3cf-a04c-4734-85f5-62ef440dd440 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:24.089854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8c3cf-a04c-4734-85f5-62ef440dd440 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:24.090163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8c3cf-a04c-4734-85f5-62ef440dd440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:24.100942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8c3cf-a04c-4734-85f5-62ef440dd440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:24.103194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8c3cf-a04c-4734-85f5-62ef440dd440 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:24.114673 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 961/1927] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:24.227772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcff4af-1cc6-4e5a-a04b-b25e85f9095e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:24.239364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcff4af-1cc6-4e5a-a04b-b25e85f9095e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:24.244502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d37de0-ad26-488c-9240-1db3fe0814d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:24.245131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d37de0-ad26-488c-9240-1db3fe0814d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:24.245435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d37de0-ad26-488c-9240-1db3fe0814d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:24.245674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d37de0-ad26-488c-9240-1db3fe0814d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:24.338032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d37de0-ad26-488c-9240-1db3fe0814d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:24.338032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d37de0-ad26-488c-9240-1db3fe0814d1 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:24.347359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 967/1928] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:26.886728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8c3cf-a04c-4734-85f5-62ef440dd440 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:26.889260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cac8c3cf-a04c-4734-85f5-62ef440dd440 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:27.014726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8c3cf-a04c-4734-85f5-62ef440dd440 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:27.021231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:27.021994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:27.022371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:27.022698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] PATH_INFO: `/v3/users/9941c3f9d9a04169b00a8f5c03d291b3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:27.024089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9941c3f9d9a04169b00a8f5c03d291b3)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:27.026272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:27.086416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:27.095007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9941c3f9d9a04169b00a8f5c03d291b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.109607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9941c3f9d9a04169b00a8f5c03d291b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.110277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9941c3f9d9a04169b00a8f5c03d291b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.110892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9941c3f9d9a04169b00a8f5c03d291b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.119859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9941c3f9d9a04169b00a8f5c03d291b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.163053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 962/1929] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:26 2022] DELETE /identity/v3/users/9941c3f9d9a04169b00a8f5c03d291b3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:27.168125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae825a4-c523-45b8-a69a-8f54ec6e0dab None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:27.168631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae825a4-c523-45b8-a69a-8f54ec6e0dab None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:27.168711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae825a4-c523-45b8-a69a-8f54ec6e0dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:27.211032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5ae825a4-c523-45b8-a69a-8f54ec6e0dab None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:27.212293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5ae825a4-c523-45b8-a69a-8f54ec6e0dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:27.352695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 968/1930] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:27.369331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:27.371204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:27.442247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570a24f-7a5b-4a57-9088-4c9f2e803cd2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:27.446595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c731b5a-573d-4f37-b4ab-0de2b1e53db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:27.447273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c731b5a-573d-4f37-b4ab-0de2b1e53db8 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:27.447641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c731b5a-573d-4f37-b4ab-0de2b1e53db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:27.447994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c731b5a-573d-4f37-b4ab-0de2b1e53db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:27.457155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c731b5a-573d-4f37-b4ab-0de2b1e53db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:27.458750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c731b5a-573d-4f37-b4ab-0de2b1e53db8 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:27.469443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 963/1931] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:27.768047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae825a4-c523-45b8-a69a-8f54ec6e0dab None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:27.769575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5ae825a4-c523-45b8-a69a-8f54ec6e0dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:27.784375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae825a4-c523-45b8-a69a-8f54ec6e0dab None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:27.793106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:27.793753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:27.794191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:27.794573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] PATH_INFO: `/v3/projects/8446d8180dbf4b438d5b26af80ea9a8d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:27.795931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8446d8180dbf4b438d5b26af80ea9a8d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:27.798045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:27.846197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8446d8180dbf4b438d5b26af80ea9a8d', 'request_id': 'req-768b7d0a-c50b-47bc-9702-96e1a252285c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.853374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8446d8180dbf4b438d5b26af80ea9a8d', 'request_id': 'req-768b7d0a-c50b-47bc-9702-96e1a252285c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.860700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8446d8180dbf4b438d5b26af80ea9a8d', 'request_id': 'req-768b7d0a-c50b-47bc-9702-96e1a252285c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.872896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] The token cache is being invalidate because project 8446d8180dbf4b438d5b26af80ea9a8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:27.873184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.873799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:27.875340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 969/1932] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:27 2022] DELETE /identity/v3/projects/8446d8180dbf4b438d5b26af80ea9a8d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:27.879429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c731b5a-573d-4f37-b4ab-0de2b1e53db8 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:27.892058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c731b5a-573d-4f37-b4ab-0de2b1e53db8 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:27.949040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:27.949676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:27.949985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:27.950265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] PATH_INFO: `/v3/users/2ee8c9d453664c5cb9eb1734875856b5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:27.951377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee8c9d453664c5cb9eb1734875856b5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:27.953280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.000507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee8c9d453664c5cb9eb1734875856b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.007974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee8c9d453664c5cb9eb1734875856b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.008608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee8c9d453664c5cb9eb1734875856b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.009378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee8c9d453664c5cb9eb1734875856b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.016651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa179a07-0a83-4d78-9c1f-cedbd239e127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee8c9d453664c5cb9eb1734875856b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.017882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:28.036512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 964/1933] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:27 2022] DELETE /identity/v3/users/2ee8c9d453664c5cb9eb1734875856b5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:28.099704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.101426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:28.116054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b7d0a-c50b-47bc-9702-96e1a252285c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.132374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:28.177173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.177816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.182269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.182545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] PATH_INFO: `/v3/users/225183cdc94c473baaf01e1cd8d2f558` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.183771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=225183cdc94c473baaf01e1cd8d2f558)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.185741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.234529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '225183cdc94c473baaf01e1cd8d2f558'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.235331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '225183cdc94c473baaf01e1cd8d2f558'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.244213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '225183cdc94c473baaf01e1cd8d2f558'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.245157 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '225183cdc94c473baaf01e1cd8d2f558'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.252844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '225183cdc94c473baaf01e1cd8d2f558'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.270597 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 970/1934] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:28 2022] DELETE /identity/v3/users/225183cdc94c473baaf01e1cd8d2f558 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:28.275840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbfef9-81f4-4499-8792-812fb8315631 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.276124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbfef9-81f4-4499-8792-812fb8315631 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.276427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbfef9-81f4-4499-8792-812fb8315631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.296178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-aacbfef9-81f4-4499-8792-812fb8315631 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:28.297383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aacbfef9-81f4-4499-8792-812fb8315631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:28.316097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.317567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:28.382101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 965/1935] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:28.394580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbb643-1160-46bf-b0ef-0c576f4cec6f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.399777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.400413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.400691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.400964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] PATH_INFO: `/v3/projects/0a4a7b485c614c779bd062be90e74a9c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.402096 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4a7b485c614c779bd062be90e74a9c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.404007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.408403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbfef9-81f4-4499-8792-812fb8315631 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.410127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aacbfef9-81f4-4499-8792-812fb8315631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:28.451870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:28.462497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4a7b485c614c779bd062be90e74a9c', 'request_id': 'req-f22470b5-2499-4e0d-913d-3c9a9940d279'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.469894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4a7b485c614c779bd062be90e74a9c', 'request_id': 'req-f22470b5-2499-4e0d-913d-3c9a9940d279'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.480246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4a7b485c614c779bd062be90e74a9c', 'request_id': 'req-f22470b5-2499-4e0d-913d-3c9a9940d279'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.481739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbfef9-81f4-4499-8792-812fb8315631 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.493800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] The token cache is being invalidate because project 0a4a7b485c614c779bd062be90e74a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:28.494153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.494857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f22470b5-2499-4e0d-913d-3c9a9940d279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.496401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 971/1936] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:28 2022] DELETE /identity/v3/projects/0a4a7b485c614c779bd062be90e74a9c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:28.501527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45727df-f173-4e13-b73c-64bc4575fb7f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.501823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45727df-f173-4e13-b73c-64bc4575fb7f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.502206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45727df-f173-4e13-b73c-64bc4575fb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.522307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d45727df-f173-4e13-b73c-64bc4575fb7f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:28.524242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d45727df-f173-4e13-b73c-64bc4575fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:28.535890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f49d7d-32ae-457d-8393-5a102160362c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.536506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f49d7d-32ae-457d-8393-5a102160362c service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.536903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f49d7d-32ae-457d-8393-5a102160362c service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.537067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f49d7d-32ae-457d-8393-5a102160362c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.538807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d1f49d7d-32ae-457d-8393-5a102160362c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:28.587459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 972/1937] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:28.590771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:28.597422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a504c-6b48-4ec0-acb2-5c2f4e4e1b52 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.597748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a504c-6b48-4ec0-acb2-5c2f4e4e1b52 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.598006 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a504c-6b48-4ec0-acb2-5c2f4e4e1b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.606402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f49d7d-32ae-457d-8393-5a102160362c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.607834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f49d7d-32ae-457d-8393-5a102160362c service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.617755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 966/1938] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:28.622078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c91a504c-6b48-4ec0-acb2-5c2f4e4e1b52 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:28.623311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c91a504c-6b48-4ec0-acb2-5c2f4e4e1b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:28.631349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 973/1939] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:28.636110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f49d7d-32ae-457d-8393-5a102160362c service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.645588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f49d7d-32ae-457d-8393-5a102160362c service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.650805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82785eaa-62b6-4b00-afec-fd4effa45ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.651414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82785eaa-62b6-4b00-afec-fd4effa45ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.651697 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82785eaa-62b6-4b00-afec-fd4effa45ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.651972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82785eaa-62b6-4b00-afec-fd4effa45ef9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.652611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82785eaa-62b6-4b00-afec-fd4effa45ef9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:28.652912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82785eaa-62b6-4b00-afec-fd4effa45ef9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.654615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82785eaa-62b6-4b00-afec-fd4effa45ef9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.668108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 967/1940] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:28:28.673459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a504c-6b48-4ec0-acb2-5c2f4e4e1b52 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.683033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a504c-6b48-4ec0-acb2-5c2f4e4e1b52 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.688217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15d244-2a68-4060-a8d1-a1499db1ee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.688824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15d244-2a68-4060-a8d1-a1499db1ee96 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.689098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15d244-2a68-4060-a8d1-a1499db1ee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.689506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15d244-2a68-4060-a8d1-a1499db1ee96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.690189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15d244-2a68-4060-a8d1-a1499db1ee96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.692065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15d244-2a68-4060-a8d1-a1499db1ee96 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.721947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 974/1941] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:28 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:28.726150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82785eaa-62b6-4b00-afec-fd4effa45ef9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.736144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82785eaa-62b6-4b00-afec-fd4effa45ef9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.798749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d96656-206a-4d54-b832-e8e92e8f04a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.799518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d96656-206a-4d54-b832-e8e92e8f04a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.799900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d96656-206a-4d54-b832-e8e92e8f04a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.800183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d96656-206a-4d54-b832-e8e92e8f04a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.800852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d96656-206a-4d54-b832-e8e92e8f04a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.802862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d96656-206a-4d54-b832-e8e92e8f04a7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.828486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 968/1942] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:28 2022] POST /identity/v3/users => generated 304 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:28.829724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15d244-2a68-4060-a8d1-a1499db1ee96 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.839948 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15d244-2a68-4060-a8d1-a1499db1ee96 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.845517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08879296-26ba-4948-8af9-fc1670913545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.846297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879296-26ba-4948-8af9-fc1670913545 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.846637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879296-26ba-4948-8af9-fc1670913545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.846975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879296-26ba-4948-8af9-fc1670913545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.847703 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08879296-26ba-4948-8af9-fc1670913545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.849355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08879296-26ba-4948-8af9-fc1670913545 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.865036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 975/1943] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:28.865223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d96656-206a-4d54-b832-e8e92e8f04a7 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.877064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d96656-206a-4d54-b832-e8e92e8f04a7 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.885224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e946ecf-8bed-4a95-a48a-e9328a8d6e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.885975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e946ecf-8bed-4a95-a48a-e9328a8d6e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.886414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e946ecf-8bed-4a95-a48a-e9328a8d6e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.886737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e946ecf-8bed-4a95-a48a-e9328a8d6e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.887777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e946ecf-8bed-4a95-a48a-e9328a8d6e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.889977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e946ecf-8bed-4a95-a48a-e9328a8d6e4f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.898882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08879296-26ba-4948-8af9-fc1670913545 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.900619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-08879296-26ba-4948-8af9-fc1670913545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:28.909733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 969/1944] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:28:28.915311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e946ecf-8bed-4a95-a48a-e9328a8d6e4f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:28.918353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08879296-26ba-4948-8af9-fc1670913545 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.925543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.926200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e946ecf-8bed-4a95-a48a-e9328a8d6e4f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:28.926466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.927775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.927957 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] PATH_INFO: `/v3/projects/250739247f8948a1b0fb7898f0238a10` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.929218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=250739247f8948a1b0fb7898f0238a10)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.931955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.932825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146692b-fdd8-4f9f-9f8b-46123e3db7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:28.933495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146692b-fdd8-4f9f-9f8b-46123e3db7ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.933786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146692b-fdd8-4f9f-9f8b-46123e3db7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.934092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146692b-fdd8-4f9f-9f8b-46123e3db7ee admin admin] PATH_INFO: `/v3/projects/cdd1c47c02b246c1b3ad6f7dc6f153db/users/1172cb24854943d187270632bebf041b/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:28.966586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146692b-fdd8-4f9f-9f8b-46123e3db7ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd1c47c02b246c1b3ad6f7dc6f153db, user_id=1172cb24854943d187270632bebf041b, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:28.969267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146692b-fdd8-4f9f-9f8b-46123e3db7ee admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:28.985197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250739247f8948a1b0fb7898f0238a10', 'request_id': 'req-21662b8f-134b-4bcf-aace-9bfbc9ff6852'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.995485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 970/1945] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:28 2022] PUT /identity/v3/projects/cdd1c47c02b246c1b3ad6f7dc6f153db/users/1172cb24854943d187270632bebf041b/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:28.995698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250739247f8948a1b0fb7898f0238a10', 'request_id': 'req-21662b8f-134b-4bcf-aace-9bfbc9ff6852'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:28.998997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080e9e2-6502-40b7-81dc-f7e17335db25 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:28.999284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080e9e2-6502-40b7-81dc-f7e17335db25 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:28.999568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080e9e2-6502-40b7-81dc-f7e17335db25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.002198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250739247f8948a1b0fb7898f0238a10', 'request_id': 'req-21662b8f-134b-4bcf-aace-9bfbc9ff6852'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.021117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] The token cache is being invalidate because project 250739247f8948a1b0fb7898f0238a10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:29.021421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.022159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.027088 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 976/1946] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:28 2022] DELETE /identity/v3/projects/250739247f8948a1b0fb7898f0238a10 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:29.028093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.029729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.034877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8080e9e2-6502-40b7-81dc-f7e17335db25 None None] MFA Rules not processed for user `1172cb24854943d187270632bebf041b`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:29.036501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8080e9e2-6502-40b7-81dc-f7e17335db25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.046702 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:29.120056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21662b8f-134b-4bcf-aace-9bfbc9ff6852 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.127346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 971/1947] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:28 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:29.142539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080e9e2-6502-40b7-81dc-f7e17335db25 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.144098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8080e9e2-6502-40b7-81dc-f7e17335db25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.164448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:29.187692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.188335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.188619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.188889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] PATH_INFO: `/v3/users/56143a362aba44bc8844537100876132` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.190035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56143a362aba44bc8844537100876132)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.191952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.213463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080e9e2-6502-40b7-81dc-f7e17335db25 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.218454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d1fbb-7479-4585-aae1-3df5ba86a0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.219066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d1fbb-7479-4585-aae1-3df5ba86a0df admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.219346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d1fbb-7479-4585-aae1-3df5ba86a0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.220039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d1fbb-7479-4585-aae1-3df5ba86a0df admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.220247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d1fbb-7479-4585-aae1-3df5ba86a0df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.222461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d1fbb-7479-4585-aae1-3df5ba86a0df admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.238002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56143a362aba44bc8844537100876132'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.239016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56143a362aba44bc8844537100876132'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.247224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56143a362aba44bc8844537100876132'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.247980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56143a362aba44bc8844537100876132'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.253927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 972/1948] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:29 2022] POST /identity/v3/projects => generated 409 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:29.254905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d1fbb-7479-4585-aae1-3df5ba86a0df admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.256561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-032d1fbb-7479-4585-aae1-3df5ba86a0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.257099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56143a362aba44bc8844537100876132'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.289880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 977/1949] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:29 2022] DELETE /identity/v3/users/56143a362aba44bc8844537100876132 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:29.334937 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d1fbb-7479-4585-aae1-3df5ba86a0df admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.394349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75531840-9f19-48ec-8642-d45ac165dbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.394965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75531840-9f19-48ec-8642-d45ac165dbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.395263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75531840-9f19-48ec-8642-d45ac165dbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.395535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75531840-9f19-48ec-8642-d45ac165dbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.396200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75531840-9f19-48ec-8642-d45ac165dbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.398128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75531840-9f19-48ec-8642-d45ac165dbb5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.424130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 973/1950] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:29 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:29.425697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.438432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12b102-5800-4f55-8bc1-e35167cd8027 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.443368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cef036-be2c-4a6a-a81d-5e246baafb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.444264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cef036-be2c-4a6a-a81d-5e246baafb7c admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.444655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cef036-be2c-4a6a-a81d-5e246baafb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.445147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cef036-be2c-4a6a-a81d-5e246baafb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.445822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cef036-be2c-4a6a-a81d-5e246baafb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.447554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cef036-be2c-4a6a-a81d-5e246baafb7c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.458236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 978/1951] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:29.462463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75531840-9f19-48ec-8642-d45ac165dbb5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.472377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75531840-9f19-48ec-8642-d45ac165dbb5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.477670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349314ba-ea8c-4a6c-b4a5-82718591450d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.478364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349314ba-ea8c-4a6c-b4a5-82718591450d admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.478646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349314ba-ea8c-4a6c-b4a5-82718591450d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.478920 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349314ba-ea8c-4a6c-b4a5-82718591450d admin admin] PATH_INFO: `/v3/projects/f40ebac4cdcd47b9a95c7a26f26ce5de/users/80aaaf433f14489281d4fbbe9ad214c3/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.495151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349314ba-ea8c-4a6c-b4a5-82718591450d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40ebac4cdcd47b9a95c7a26f26ce5de, user_id=80aaaf433f14489281d4fbbe9ad214c3, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.497801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349314ba-ea8c-4a6c-b4a5-82718591450d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.511536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 974/1952] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:29 2022] PUT /identity/v3/projects/f40ebac4cdcd47b9a95c7a26f26ce5de/users/80aaaf433f14489281d4fbbe9ad214c3/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:29.515567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cef036-be2c-4a6a-a81d-5e246baafb7c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.526805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cef036-be2c-4a6a-a81d-5e246baafb7c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.560832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:29.561993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349314ba-ea8c-4a6c-b4a5-82718591450d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.564089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-349314ba-ea8c-4a6c-b4a5-82718591450d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.606599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:29.610677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca32b75-5a6e-4369-b4b2-e41ac147bf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.611201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca32b75-5a6e-4369-b4b2-e41ac147bf92 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.611947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca32b75-5a6e-4369-b4b2-e41ac147bf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.612065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca32b75-5a6e-4369-b4b2-e41ac147bf92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.612721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca32b75-5a6e-4369-b4b2-e41ac147bf92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.614506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca32b75-5a6e-4369-b4b2-e41ac147bf92 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.629986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 979/1953] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:29.632017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca32b75-5a6e-4369-b4b2-e41ac147bf92 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.642205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca32b75-5a6e-4369-b4b2-e41ac147bf92 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.646993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac343b2-8a7f-4233-893f-fb7863ddf098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.647655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac343b2-8a7f-4233-893f-fb7863ddf098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.647936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac343b2-8a7f-4233-893f-fb7863ddf098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.648211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac343b2-8a7f-4233-893f-fb7863ddf098 admin admin] PATH_INFO: `/v3/projects/f40ebac4cdcd47b9a95c7a26f26ce5de/users/80aaaf433f14489281d4fbbe9ad214c3/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.650665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac343b2-8a7f-4233-893f-fb7863ddf098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40ebac4cdcd47b9a95c7a26f26ce5de, user_id=80aaaf433f14489281d4fbbe9ad214c3, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.653303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac343b2-8a7f-4233-893f-fb7863ddf098 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.666532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 980/1954] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:29 2022] PUT /identity/v3/projects/f40ebac4cdcd47b9a95c7a26f26ce5de/users/80aaaf433f14489281d4fbbe9ad214c3/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:29.669348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349314ba-ea8c-4a6c-b4a5-82718591450d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.677619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea49cd-31d5-4687-bfd8-ad13657e06fb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.677902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea49cd-31d5-4687-bfd8-ad13657e06fb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.678203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea49cd-31d5-4687-bfd8-ad13657e06fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.707024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2aea49cd-31d5-4687-bfd8-ad13657e06fb None None] MFA Rules not processed for user `80aaaf433f14489281d4fbbe9ad214c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:29.708241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2aea49cd-31d5-4687-bfd8-ad13657e06fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.733666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.734346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.734749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.735030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] PATH_INFO: `/v3/projects/64a36f687dfe40cb96db29a9ddba51aa` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.736153 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a36f687dfe40cb96db29a9ddba51aa)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.738407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.786041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a36f687dfe40cb96db29a9ddba51aa', 'request_id': 'req-d72b73ad-3121-47b5-8ccc-35396014f323'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.791134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 981/1955] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:29 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:29.795248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a36f687dfe40cb96db29a9ddba51aa', 'request_id': 'req-d72b73ad-3121-47b5-8ccc-35396014f323'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.801434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a36f687dfe40cb96db29a9ddba51aa', 'request_id': 'req-d72b73ad-3121-47b5-8ccc-35396014f323'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.803043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea49cd-31d5-4687-bfd8-ad13657e06fb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.804543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2aea49cd-31d5-4687-bfd8-ad13657e06fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.813567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] The token cache is being invalidate because project 64a36f687dfe40cb96db29a9ddba51aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:29.813916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.814609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d72b73ad-3121-47b5-8ccc-35396014f323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:29.816114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 975/1956] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:29 2022] DELETE /identity/v3/projects/64a36f687dfe40cb96db29a9ddba51aa => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:29.820737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e22e9-3574-4488-b9f7-d05a74c6a4a8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.821030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e22e9-3574-4488-b9f7-d05a74c6a4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.821311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e22e9-3574-4488-b9f7-d05a74c6a4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.840730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-991e22e9-3574-4488-b9f7-d05a74c6a4a8 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:29.841948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-991e22e9-3574-4488-b9f7-d05a74c6a4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.862564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea49cd-31d5-4687-bfd8-ad13657e06fb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.890460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:29.907058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 976/1957] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:29.915547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b1832-2324-41a7-aa52-369a0f7fe4b0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.916113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29ba73-65fb-40c6-88b2-56d79d46a3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.916246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b1832-2324-41a7-aa52-369a0f7fe4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.916246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b1832-2324-41a7-aa52-369a0f7fe4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.916429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29ba73-65fb-40c6-88b2-56d79d46a3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.916429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29ba73-65fb-40c6-88b2-56d79d46a3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.916681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29ba73-65fb-40c6-88b2-56d79d46a3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.918436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-aa29ba73-65fb-40c6-88b2-56d79d46a3e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.925098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:29.935355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-408b1832-2324-41a7-aa52-369a0f7fe4b0 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:29.936542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-408b1832-2324-41a7-aa52-369a0f7fe4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:29.944401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 977/1958] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:29.949145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b1832-2324-41a7-aa52-369a0f7fe4b0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.957844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b1832-2324-41a7-aa52-369a0f7fe4b0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.963931 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40175f48-5025-4919-b10f-5b29307273f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:29.964528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40175f48-5025-4919-b10f-5b29307273f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:29.964806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40175f48-5025-4919-b10f-5b29307273f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:29.965082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40175f48-5025-4919-b10f-5b29307273f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:29.965706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40175f48-5025-4919-b10f-5b29307273f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:29.966094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40175f48-5025-4919-b10f-5b29307273f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.967666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40175f48-5025-4919-b10f-5b29307273f2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.980679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 978/1959] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:28:29.984991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40175f48-5025-4919-b10f-5b29307273f2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:29.986105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29ba73-65fb-40c6-88b2-56d79d46a3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:29.987813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29ba73-65fb-40c6-88b2-56d79d46a3e6 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:29.996756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40175f48-5025-4919-b10f-5b29307273f2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:29.997525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 982/1960] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:29 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:30.002492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966b91e-0680-4edc-b244-4c17b4a78870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.003111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966b91e-0680-4edc-b244-4c17b4a78870 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.003399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966b91e-0680-4edc-b244-4c17b4a78870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.004125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966b91e-0680-4edc-b244-4c17b4a78870 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.004303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966b91e-0680-4edc-b244-4c17b4a78870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.006478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966b91e-0680-4edc-b244-4c17b4a78870 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.045722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 979/1961] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:29 2022] POST /identity/v3/projects => generated 393 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:30.046860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29ba73-65fb-40c6-88b2-56d79d46a3e6 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.057365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29ba73-65fb-40c6-88b2-56d79d46a3e6 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.135754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac614e-83a4-4d9d-bad9-b2fa696ed8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.136729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac614e-83a4-4d9d-bad9-b2fa696ed8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.137230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac614e-83a4-4d9d-bad9-b2fa696ed8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.138049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac614e-83a4-4d9d-bad9-b2fa696ed8a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.139188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac614e-83a4-4d9d-bad9-b2fa696ed8a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.142698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac614e-83a4-4d9d-bad9-b2fa696ed8a2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.186865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 983/1962] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:30 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:30.191698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966b91e-0680-4edc-b244-4c17b4a78870 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.191830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac614e-83a4-4d9d-bad9-b2fa696ed8a2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.193651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5eac614e-83a4-4d9d-bad9-b2fa696ed8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:30.205392 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966b91e-0680-4edc-b244-4c17b4a78870 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.222076 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db4ace-7473-4d9e-bf62-3c33929c1807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.222843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db4ace-7473-4d9e-bf62-3c33929c1807 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.223204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db4ace-7473-4d9e-bf62-3c33929c1807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.223677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db4ace-7473-4d9e-bf62-3c33929c1807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.224550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db4ace-7473-4d9e-bf62-3c33929c1807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.226201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db4ace-7473-4d9e-bf62-3c33929c1807 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.239659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 980/1963] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:30.240706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db4ace-7473-4d9e-bf62-3c33929c1807 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.250457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db4ace-7473-4d9e-bf62-3c33929c1807 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.255026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44980e9e-ae21-4394-b1d6-2ee2bb46737a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.255496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac614e-83a4-4d9d-bad9-b2fa696ed8a2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.255602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44980e9e-ae21-4394-b1d6-2ee2bb46737a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.255868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44980e9e-ae21-4394-b1d6-2ee2bb46737a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.256139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44980e9e-ae21-4394-b1d6-2ee2bb46737a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.256756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44980e9e-ae21-4394-b1d6-2ee2bb46737a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.258981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44980e9e-ae21-4394-b1d6-2ee2bb46737a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.259690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484593a-183b-4903-a66e-0b2b5737e2da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.260285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484593a-183b-4903-a66e-0b2b5737e2da service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.260564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484593a-183b-4903-a66e-0b2b5737e2da service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.260831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484593a-183b-4903-a66e-0b2b5737e2da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.262566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4484593a-183b-4903-a66e-0b2b5737e2da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:30.269027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 981/1964] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:30.273110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44980e9e-ae21-4394-b1d6-2ee2bb46737a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.283099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44980e9e-ae21-4394-b1d6-2ee2bb46737a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.287760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faeae5-11d0-4494-8c84-3d1d8ae0dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.288391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faeae5-11d0-4494-8c84-3d1d8ae0dfcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.288938 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faeae5-11d0-4494-8c84-3d1d8ae0dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.289254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faeae5-11d0-4494-8c84-3d1d8ae0dfcf admin admin] PATH_INFO: `/v3/projects/13e2283793b347449e409e8e44ea3db8/users/c0007ace41664869819f1f4b901c0e2a/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.305367 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faeae5-11d0-4494-8c84-3d1d8ae0dfcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e2283793b347449e409e8e44ea3db8, user_id=c0007ace41664869819f1f4b901c0e2a, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.308055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faeae5-11d0-4494-8c84-3d1d8ae0dfcf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.310578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:30.323368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 982/1965] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:30 2022] PUT /identity/v3/projects/13e2283793b347449e409e8e44ea3db8/users/c0007ace41664869819f1f4b901c0e2a/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:30.326093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484593a-183b-4903-a66e-0b2b5737e2da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.327145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484593a-183b-4903-a66e-0b2b5737e2da service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.333738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f1627-cc45-4126-8b6f-1637730d6d4e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.334055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f1627-cc45-4126-8b6f-1637730d6d4e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.334332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f1627-cc45-4126-8b6f-1637730d6d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.336364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 984/1966] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:30 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:30.362468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-867f1627-cc45-4126-8b6f-1637730d6d4e None None] MFA Rules not processed for user `c0007ace41664869819f1f4b901c0e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:30.363677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-867f1627-cc45-4126-8b6f-1637730d6d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:30.449447 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 983/1967] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:30.454213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484593a-183b-4903-a66e-0b2b5737e2da service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.466268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484593a-183b-4903-a66e-0b2b5737e2da service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.532062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37800857-3873-4841-9e9e-9e824816e39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.532669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37800857-3873-4841-9e9e-9e824816e39f admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.532945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37800857-3873-4841-9e9e-9e824816e39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.533214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37800857-3873-4841-9e9e-9e824816e39f admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.533863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37800857-3873-4841-9e9e-9e824816e39f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.536067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37800857-3873-4841-9e9e-9e824816e39f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.565612 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 985/1968] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:30 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:30.569764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f1627-cc45-4126-8b6f-1637730d6d4e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.582208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f1627-cc45-4126-8b6f-1637730d6d4e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.656698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:30.673057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25b233-16b8-4882-ab2e-9d2ee2377dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.673644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25b233-16b8-4882-ab2e-9d2ee2377dea admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.673923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25b233-16b8-4882-ab2e-9d2ee2377dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.674229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25b233-16b8-4882-ab2e-9d2ee2377dea admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.674859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25b233-16b8-4882-ab2e-9d2ee2377dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.676884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25b233-16b8-4882-ab2e-9d2ee2377dea admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.698521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 984/1969] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:30 2022] POST /identity/v3/users => generated 301 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:30.703114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37800857-3873-4841-9e9e-9e824816e39f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.713923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37800857-3873-4841-9e9e-9e824816e39f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.718665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c908d-37e1-4416-8eed-c7d535cad817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.719261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c908d-37e1-4416-8eed-c7d535cad817 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.719538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c908d-37e1-4416-8eed-c7d535cad817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.719811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c908d-37e1-4416-8eed-c7d535cad817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.720467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c908d-37e1-4416-8eed-c7d535cad817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.722086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c908d-37e1-4416-8eed-c7d535cad817 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.736889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 986/1970] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:30.737999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25b233-16b8-4882-ab2e-9d2ee2377dea admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.748597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25b233-16b8-4882-ab2e-9d2ee2377dea admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.753137 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c0b0b-f447-4d22-8953-ed22fdd812ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.753792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c0b0b-f447-4d22-8953-ed22fdd812ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.754166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c0b0b-f447-4d22-8953-ed22fdd812ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.754410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c0b0b-f447-4d22-8953-ed22fdd812ce admin admin] PATH_INFO: `/v3/projects/e652b824dc044f8e9c35e85e7697f93d/users/e0ba3b7cf58448bbab8d3cbc963eb8fa/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.770414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c0b0b-f447-4d22-8953-ed22fdd812ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e652b824dc044f8e9c35e85e7697f93d, user_id=e0ba3b7cf58448bbab8d3cbc963eb8fa, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.773048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c0b0b-f447-4d22-8953-ed22fdd812ce admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.785475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 985/1971] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:30 2022] PUT /identity/v3/projects/e652b824dc044f8e9c35e85e7697f93d/users/e0ba3b7cf58448bbab8d3cbc963eb8fa/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:30.790550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c908d-37e1-4416-8eed-c7d535cad817 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.800627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c908d-37e1-4416-8eed-c7d535cad817 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.860494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf2470-ae45-4737-bc67-53e0e9852ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.861112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf2470-ae45-4737-bc67-53e0e9852ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.861647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf2470-ae45-4737-bc67-53e0e9852ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.861924 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf2470-ae45-4737-bc67-53e0e9852ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.862669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf2470-ae45-4737-bc67-53e0e9852ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.864298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf2470-ae45-4737-bc67-53e0e9852ce5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.875020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 987/1972] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:30.878959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c0b0b-f447-4d22-8953-ed22fdd812ce admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:30.889228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c0b0b-f447-4d22-8953-ed22fdd812ce admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:30.893776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcaa63f-f000-4a9d-9528-db29f3253cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:30.894507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaa63f-f000-4a9d-9528-db29f3253cec admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.894783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaa63f-f000-4a9d-9528-db29f3253cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.895055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaa63f-f000-4a9d-9528-db29f3253cec admin admin] PATH_INFO: `/v3/projects/e652b824dc044f8e9c35e85e7697f93d/users/e0ba3b7cf58448bbab8d3cbc963eb8fa/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.896938 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcaa63f-f000-4a9d-9528-db29f3253cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e652b824dc044f8e9c35e85e7697f93d, user_id=e0ba3b7cf58448bbab8d3cbc963eb8fa, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:30.899670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcaa63f-f000-4a9d-9528-db29f3253cec admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:30.911245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 986/1973] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:30 2022] PUT /identity/v3/projects/e652b824dc044f8e9c35e85e7697f93d/users/e0ba3b7cf58448bbab8d3cbc963eb8fa/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:30.923228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1b70-8e1f-4022-93a9-e3873b92c40b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:30.923518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1b70-8e1f-4022-93a9-e3873b92c40b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:30.923795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1b70-8e1f-4022-93a9-e3873b92c40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:30.953350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-693d1b70-8e1f-4022-93a9-e3873b92c40b None None] MFA Rules not processed for user `e0ba3b7cf58448bbab8d3cbc963eb8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:30.954645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-693d1b70-8e1f-4022-93a9-e3873b92c40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:31.002320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:31.028017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 988/1974] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:31.040316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcaa63f-f000-4a9d-9528-db29f3253cec admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:31.102688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcaa63f-f000-4a9d-9528-db29f3253cec admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:31.106402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583b8fa-2625-4359-863e-9f3677440fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:31.107012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583b8fa-2625-4359-863e-9f3677440fde service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:31.107335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583b8fa-2625-4359-863e-9f3677440fde service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:31.107560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583b8fa-2625-4359-863e-9f3677440fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:31.128459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583b8fa-2625-4359-863e-9f3677440fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:31.129973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583b8fa-2625-4359-863e-9f3677440fde service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:31.140228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 987/1975] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:31.199199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1b70-8e1f-4022-93a9-e3873b92c40b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:31.209675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1b70-8e1f-4022-93a9-e3873b92c40b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:31.214197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84829d-7e09-4c0a-9784-50d00968588d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:31.214800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84829d-7e09-4c0a-9784-50d00968588d service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:31.215081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84829d-7e09-4c0a-9784-50d00968588d service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:31.215354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84829d-7e09-4c0a-9784-50d00968588d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:31.277887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84829d-7e09-4c0a-9784-50d00968588d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:31.279336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84829d-7e09-4c0a-9784-50d00968588d service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:31.287923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 989/1976] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:38.871641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583b8fa-2625-4359-863e-9f3677440fde service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:38.948206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583b8fa-2625-4359-863e-9f3677440fde service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:38.952773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:38.953637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:38.954001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:38.954275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] PATH_INFO: `/v3/users/c0007ace41664869819f1f4b901c0e2a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:38.955483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0007ace41664869819f1f4b901c0e2a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:38.957388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:39.006465 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0007ace41664869819f1f4b901c0e2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.010650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:39.015951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0007ace41664869819f1f4b901c0e2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.016671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0007ace41664869819f1f4b901c0e2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.017347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0007ace41664869819f1f4b901c0e2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.034896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0007ace41664869819f1f4b901c0e2a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.064462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 988/1977] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:38 2022] DELETE /identity/v3/users/c0007ace41664869819f1f4b901c0e2a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:39.066729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ba154-9c32-46a8-b3be-e9afb6718b57 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:39.067240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ba154-9c32-46a8-b3be-e9afb6718b57 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:39.067441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ba154-9c32-46a8-b3be-e9afb6718b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:39.093444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-dd0ba154-9c32-46a8-b3be-e9afb6718b57 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:39.096129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dd0ba154-9c32-46a8-b3be-e9afb6718b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:39.212095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 990/1978] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:39.223235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:39.224884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:39.302510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2caae-4a50-41b9-9d60-d0da1547e490 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:39.306785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de033ec-7866-4d51-9958-09ebb7c0acb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:39.307496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de033ec-7866-4d51-9958-09ebb7c0acb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:39.308009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de033ec-7866-4d51-9958-09ebb7c0acb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:39.308298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de033ec-7866-4d51-9958-09ebb7c0acb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:39.319520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de033ec-7866-4d51-9958-09ebb7c0acb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:39.321033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de033ec-7866-4d51-9958-09ebb7c0acb4 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:39.331271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 989/1979] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:39.654373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ba154-9c32-46a8-b3be-e9afb6718b57 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:39.657145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dd0ba154-9c32-46a8-b3be-e9afb6718b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:39.671190 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ba154-9c32-46a8-b3be-e9afb6718b57 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:39.676573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:39.677291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:39.677569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:39.677837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] PATH_INFO: `/v3/projects/13e2283793b347449e409e8e44ea3db8` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:39.679017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e2283793b347449e409e8e44ea3db8)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:39.681070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:39.738261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e2283793b347449e409e8e44ea3db8', 'request_id': 'req-bda32af3-7d95-4e87-afdb-0821927e874e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.750298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e2283793b347449e409e8e44ea3db8', 'request_id': 'req-bda32af3-7d95-4e87-afdb-0821927e874e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.761387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e2283793b347449e409e8e44ea3db8', 'request_id': 'req-bda32af3-7d95-4e87-afdb-0821927e874e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.775766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] The token cache is being invalidate because project 13e2283793b347449e409e8e44ea3db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:39.776460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.778383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.784865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 991/1980] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:39 2022] DELETE /identity/v3/projects/13e2283793b347449e409e8e44ea3db8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:39.786734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de033ec-7866-4d51-9958-09ebb7c0acb4 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:39.798324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de033ec-7866-4d51-9958-09ebb7c0acb4 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:39.860013 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:39.860656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:39.860938 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:39.861209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] PATH_INFO: `/v3/users/e0ba3b7cf58448bbab8d3cbc963eb8fa` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:39.862662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ba3b7cf58448bbab8d3cbc963eb8fa)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:39.864583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:39.914679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ba3b7cf58448bbab8d3cbc963eb8fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.922498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ba3b7cf58448bbab8d3cbc963eb8fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.923122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ba3b7cf58448bbab8d3cbc963eb8fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.923754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ba3b7cf58448bbab8d3cbc963eb8fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.931350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-06b0f1c1-a834-4a10-aadc-659c8b8c33ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ba3b7cf58448bbab8d3cbc963eb8fa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:39.940644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:39.954468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 990/1981] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:39 2022] DELETE /identity/v3/users/e0ba3b7cf58448bbab8d3cbc963eb8fa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:40.231258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:40.232939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:40.246152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:40.250955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda32af3-7d95-4e87-afdb-0821927e874e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:40.322585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:40.323229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.323510 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.323929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] PATH_INFO: `/v3/projects/e652b824dc044f8e9c35e85e7697f93d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.325067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=e652b824dc044f8e9c35e85e7697f93d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:40.327040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:40.374473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e652b824dc044f8e9c35e85e7697f93d', 'request_id': 'req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.380465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e652b824dc044f8e9c35e85e7697f93d', 'request_id': 'req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.388854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e652b824dc044f8e9c35e85e7697f93d', 'request_id': 'req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.400083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] The token cache is being invalidate because project e652b824dc044f8e9c35e85e7697f93d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:40.400374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.401007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5d48f7b2-7cd3-4479-bf8a-66227d1e8eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.402541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 992/1982] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:40 2022] DELETE /identity/v3/projects/e652b824dc044f8e9c35e85e7697f93d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:40.407368 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb32e9-bff0-44ec-8449-a16f814929dc None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.407650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb32e9-bff0-44ec-8449-a16f814929dc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.407920 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb32e9-bff0-44ec-8449-a16f814929dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.427916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f1bb32e9-bff0-44ec-8449-a16f814929dc None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:40.429085 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f1bb32e9-bff0-44ec-8449-a16f814929dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:40.488084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 991/1983] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:40.498363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04675f4-dfe8-4023-82c6-28401cf69976 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.498645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04675f4-dfe8-4023-82c6-28401cf69976 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.498920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04675f4-dfe8-4023-82c6-28401cf69976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.520063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e04675f4-dfe8-4023-82c6-28401cf69976 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:40.521261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e04675f4-dfe8-4023-82c6-28401cf69976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:40.532251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 993/1984] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:40.536283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb32e9-bff0-44ec-8449-a16f814929dc None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:40.547591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb32e9-bff0-44ec-8449-a16f814929dc None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:40.552529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139bff8-eab3-49da-a28e-4d72cff086ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:40.553275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139bff8-eab3-49da-a28e-4d72cff086ab admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.554691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139bff8-eab3-49da-a28e-4d72cff086ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.554691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139bff8-eab3-49da-a28e-4d72cff086ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.554691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139bff8-eab3-49da-a28e-4d72cff086ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:40.554950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139bff8-eab3-49da-a28e-4d72cff086ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:40.557511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139bff8-eab3-49da-a28e-4d72cff086ab admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:40.571252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 992/1985] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:28:40.576411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04675f4-dfe8-4023-82c6-28401cf69976 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:40.590580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04675f4-dfe8-4023-82c6-28401cf69976 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:40.597735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17514cdf-ac12-4975-a706-79f49d5606ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:40.598772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17514cdf-ac12-4975-a706-79f49d5606ae admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.599673 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17514cdf-ac12-4975-a706-79f49d5606ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.600100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17514cdf-ac12-4975-a706-79f49d5606ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.600917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17514cdf-ac12-4975-a706-79f49d5606ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:40.604174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17514cdf-ac12-4975-a706-79f49d5606ae admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:40.652651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 994/1986] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:40 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:40.659071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139bff8-eab3-49da-a28e-4d72cff086ab admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:40.670688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139bff8-eab3-49da-a28e-4d72cff086ab admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:40.690209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17514cdf-ac12-4975-a706-79f49d5606ae admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:40.691932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-17514cdf-ac12-4975-a706-79f49d5606ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:40.728895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14357db6-cc05-4fb3-829d-e9aef77d9934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:40.729502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14357db6-cc05-4fb3-829d-e9aef77d9934 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.729787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14357db6-cc05-4fb3-829d-e9aef77d9934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.730095 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14357db6-cc05-4fb3-829d-e9aef77d9934 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.730725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14357db6-cc05-4fb3-829d-e9aef77d9934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:40.732675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14357db6-cc05-4fb3-829d-e9aef77d9934 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:40.758196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 993/1987] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:40 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:40.759104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17514cdf-ac12-4975-a706-79f49d5606ae admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:40.759498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14357db6-cc05-4fb3-829d-e9aef77d9934 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:40.764077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:40.764753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.764973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.765246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] PATH_INFO: `/v3/users/c93a57f357554554ac5d92c1b1d836ec` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.766531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93a57f357554554ac5d92c1b1d836ec)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:40.768750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:40.769853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14357db6-cc05-4fb3-829d-e9aef77d9934 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:40.774960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:40.776138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7126b2-e621-46e5-a463-3e5253f0a024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:40.777001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7126b2-e621-46e5-a463-3e5253f0a024 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.777273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7126b2-e621-46e5-a463-3e5253f0a024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.777550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7126b2-e621-46e5-a463-3e5253f0a024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.778277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7126b2-e621-46e5-a463-3e5253f0a024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:40.780529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7126b2-e621-46e5-a463-3e5253f0a024 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:40.800162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 994/1988] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:40.801431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7126b2-e621-46e5-a463-3e5253f0a024 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:40.815350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7126b2-e621-46e5-a463-3e5253f0a024 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:40.820756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c9429-edd1-4a3f-b451-6119326fd6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:40.821388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c9429-edd1-4a3f-b451-6119326fd6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.821775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c9429-edd1-4a3f-b451-6119326fd6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.821995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c9429-edd1-4a3f-b451-6119326fd6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.822668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c9429-edd1-4a3f-b451-6119326fd6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:40.824377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c9429-edd1-4a3f-b451-6119326fd6e4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:40.826810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93a57f357554554ac5d92c1b1d836ec'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.827485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93a57f357554554ac5d92c1b1d836ec'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.835375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 995/1989] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:40.840650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c9429-edd1-4a3f-b451-6119326fd6e4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:40.840781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93a57f357554554ac5d92c1b1d836ec'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.841368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93a57f357554554ac5d92c1b1d836ec'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.849565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3064d88e-6733-4331-bec4-9b10f23f2ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93a57f357554554ac5d92c1b1d836ec'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:40.852369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c9429-edd1-4a3f-b451-6119326fd6e4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:40.857694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef259f1a-33dd-4881-a01b-cd1074ce3a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:40.858701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef259f1a-33dd-4881-a01b-cd1074ce3a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.858897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef259f1a-33dd-4881-a01b-cd1074ce3a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.859231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef259f1a-33dd-4881-a01b-cd1074ce3a10 admin admin] PATH_INFO: `/v3/projects/ba8fd917635c4153b0e7d73c38e687fa/users/ea890d42ec1e4b0682ee7a19a016b130/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.871813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 995/1990] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:40 2022] DELETE /identity/v3/users/c93a57f357554554ac5d92c1b1d836ec => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:40.876061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef259f1a-33dd-4881-a01b-cd1074ce3a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba8fd917635c4153b0e7d73c38e687fa, user_id=ea890d42ec1e4b0682ee7a19a016b130, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:40.876174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f3ff8-4ff2-4e52-9db8-b25c34d3fb1a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.876346 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f3ff8-4ff2-4e52-9db8-b25c34d3fb1a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.876604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f3ff8-4ff2-4e52-9db8-b25c34d3fb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.879102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef259f1a-33dd-4881-a01b-cd1074ce3a10 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:40.895239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 996/1991] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:40 2022] PUT /identity/v3/projects/ba8fd917635c4153b0e7d73c38e687fa/users/ea890d42ec1e4b0682ee7a19a016b130/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:40.898851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4b6f3ff8-4ff2-4e52-9db8-b25c34d3fb1a None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:40.900839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4b6f3ff8-4ff2-4e52-9db8-b25c34d3fb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:40.912285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e8c10-80b0-49d1-a5e1-bd34d917ba9d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:40.912662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e8c10-80b0-49d1-a5e1-bd34d917ba9d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:40.913037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e8c10-80b0-49d1-a5e1-bd34d917ba9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:40.952034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b59e8c10-80b0-49d1-a5e1-bd34d917ba9d None None] MFA Rules not processed for user `ea890d42ec1e4b0682ee7a19a016b130`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:40.953514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b59e8c10-80b0-49d1-a5e1-bd34d917ba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:40.964058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:40.967765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 996/1992] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:40.979746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f3ff8-4ff2-4e52-9db8-b25c34d3fb1a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:40.981669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4b6f3ff8-4ff2-4e52-9db8-b25c34d3fb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:41.027583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 997/1993] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:41.038161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e8c10-80b0-49d1-a5e1-bd34d917ba9d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:41.039578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b59e8c10-80b0-49d1-a5e1-bd34d917ba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:41.039992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f3ff8-4ff2-4e52-9db8-b25c34d3fb1a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:41.044140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5c8b7-4a98-41c9-86fb-26884c0d3a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:41.044745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5c8b7-4a98-41c9-86fb-26884c0d3a1b service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:41.045027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5c8b7-4a98-41c9-86fb-26884c0d3a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:41.045298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5c8b7-4a98-41c9-86fb-26884c0d3a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:41.054811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5c8b7-4a98-41c9-86fb-26884c0d3a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:41.056366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5c8b7-4a98-41c9-86fb-26884c0d3a1b service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:41.066217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 997/1994] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:41.099365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e8c10-80b0-49d1-a5e1-bd34d917ba9d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:41.103955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf71b4-002e-4c89-a38b-b9cd864f1303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:41.104574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf71b4-002e-4c89-a38b-b9cd864f1303 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:41.104854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf71b4-002e-4c89-a38b-b9cd864f1303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:41.105124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf71b4-002e-4c89-a38b-b9cd864f1303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:41.114462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf71b4-002e-4c89-a38b-b9cd864f1303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:41.115988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf71b4-002e-4c89-a38b-b9cd864f1303 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:41.125595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 998/1995] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:41.419180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5c8b7-4a98-41c9-86fb-26884c0d3a1b service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:41.421213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-18f5c8b7-4a98-41c9-86fb-26884c0d3a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:41.437616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5c8b7-4a98-41c9-86fb-26884c0d3a1b service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:41.443696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:41.444542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:41.444914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:41.445232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] PATH_INFO: `/v3/projects/196a0dec4e2a4cdeb67d3b7496a5d982` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:41.446861 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] RBAC: Authorizing `identity:delete_project(project_id=196a0dec4e2a4cdeb67d3b7496a5d982)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:41.448922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:41.497165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:41.548578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '196a0dec4e2a4cdeb67d3b7496a5d982', 'request_id': 'req-8053ef20-88d2-4fe4-b7c8-2e46d982689c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.556689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '196a0dec4e2a4cdeb67d3b7496a5d982', 'request_id': 'req-8053ef20-88d2-4fe4-b7c8-2e46d982689c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.565096 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '196a0dec4e2a4cdeb67d3b7496a5d982', 'request_id': 'req-8053ef20-88d2-4fe4-b7c8-2e46d982689c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.580376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] The token cache is being invalidate because project 196a0dec4e2a4cdeb67d3b7496a5d982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:41.580668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.581331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.582931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 998/1996] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:41 2022] DELETE /identity/v3/projects/196a0dec4e2a4cdeb67d3b7496a5d982 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:41.591616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf71b4-002e-4c89-a38b-b9cd864f1303 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:41.606332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf71b4-002e-4c89-a38b-b9cd864f1303 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:41.662185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:41.662810 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:41.663163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:41.663431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] PATH_INFO: `/v3/users/c74a451b0043491996c5f523d1afab22` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:41.664624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c74a451b0043491996c5f523d1afab22)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:41.666758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:41.718540 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c74a451b0043491996c5f523d1afab22'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.726365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c74a451b0043491996c5f523d1afab22'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.726980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c74a451b0043491996c5f523d1afab22'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.727611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c74a451b0043491996c5f523d1afab22'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.729642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:41.735660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c74a451b0043491996c5f523d1afab22'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:41.755992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 999/1997] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:41 2022] DELETE /identity/v3/users/c74a451b0043491996c5f523d1afab22 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:42.055312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.058242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:42.075387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053ef20-88d2-4fe4-b7c8-2e46d982689c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:42.169541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.172500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:42.173540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.173820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.174140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] PATH_INFO: `/v3/projects/da9afbf3d2a54593bb1daf3d0237a44c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.175489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:42.175588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9afbf3d2a54593bb1daf3d0237a44c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:42.178070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:42.207689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c4e6b-1a71-47a7-b527-1207ecb9fef3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:42.216114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:42.216814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.217101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.217381 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] PATH_INFO: `/v3/users/1172cb24854943d187270632bebf041b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.223061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1172cb24854943d187270632bebf041b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:42.225430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:42.242087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9afbf3d2a54593bb1daf3d0237a44c', 'request_id': 'req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.249451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9afbf3d2a54593bb1daf3d0237a44c', 'request_id': 'req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.257839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9afbf3d2a54593bb1daf3d0237a44c', 'request_id': 'req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.269656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] The token cache is being invalidate because project da9afbf3d2a54593bb1daf3d0237a44c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:42.270086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.270991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3bd3a4a-5b08-47b6-9c10-c16aa6177faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.272413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 999/1998] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:42 2022] DELETE /identity/v3/projects/da9afbf3d2a54593bb1daf3d0237a44c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:42.276583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1172cb24854943d187270632bebf041b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.277668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81364050-909a-4eae-8434-948cb521c3b1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.277970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81364050-909a-4eae-8434-948cb521c3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.278250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81364050-909a-4eae-8434-948cb521c3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.284314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1172cb24854943d187270632bebf041b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.285122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1172cb24854943d187270632bebf041b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.285916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1172cb24854943d187270632bebf041b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.294552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e24f8cf3-e0b3-4c5e-b0ea-28962a356525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1172cb24854943d187270632bebf041b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:42.299881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-81364050-909a-4eae-8434-948cb521c3b1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:42.301149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-81364050-909a-4eae-8434-948cb521c3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:42.312643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1000/1999] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:42 2022] DELETE /identity/v3/users/1172cb24854943d187270632bebf041b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:42.316910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c0492-ce5e-4bc4-940f-c722e5eb8c37 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.317202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c0492-ce5e-4bc4-940f-c722e5eb8c37 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.317484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c0492-ce5e-4bc4-940f-c722e5eb8c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.338440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3d8c0492-ce5e-4bc4-940f-c722e5eb8c37 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:42.339701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3d8c0492-ce5e-4bc4-940f-c722e5eb8c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:42.341102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:42.376623 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1000/2000] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:42.378041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fd2f5-bf27-4751-8e30-4f1f3cd36ebe None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.378336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fd2f5-bf27-4751-8e30-4f1f3cd36ebe None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.378614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fd2f5-bf27-4751-8e30-4f1f3cd36ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.400394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8f8fd2f5-bf27-4751-8e30-4f1f3cd36ebe None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:42.401647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8f8fd2f5-bf27-4751-8e30-4f1f3cd36ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:42.416177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1001/2001] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:42.427818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c0492-ce5e-4bc4-940f-c722e5eb8c37 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.429221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3d8c0492-ce5e-4bc4-940f-c722e5eb8c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:42.474412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1001/2002] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:42.478859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fd2f5-bf27-4751-8e30-4f1f3cd36ebe None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.489701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fd2f5-bf27-4751-8e30-4f1f3cd36ebe None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:42.493727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:42.499399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b7406-3597-478e-8c4e-01b1a6a70c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:42.500108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b7406-3597-478e-8c4e-01b1a6a70c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.500457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b7406-3597-478e-8c4e-01b1a6a70c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.500754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c0492-ce5e-4bc4-940f-c722e5eb8c37 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:42.500885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b7406-3597-478e-8c4e-01b1a6a70c9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.501580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b7406-3597-478e-8c4e-01b1a6a70c9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:42.501977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b7406-3597-478e-8c4e-01b1a6a70c9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:42.503865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b7406-3597-478e-8c4e-01b1a6a70c9b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:42.505719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df96623-4f00-432a-aaea-20816bcdff2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:42.506910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df96623-4f00-432a-aaea-20816bcdff2f service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.507461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df96623-4f00-432a-aaea-20816bcdff2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.510246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df96623-4f00-432a-aaea-20816bcdff2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.517708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1002/2003] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:28:42.521084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df96623-4f00-432a-aaea-20816bcdff2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:42.523056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df96623-4f00-432a-aaea-20816bcdff2f service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:42.526899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b7406-3597-478e-8c4e-01b1a6a70c9b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.535325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1002/2004] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:42.552746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b7406-3597-478e-8c4e-01b1a6a70c9b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:42.577166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90011f8a-3e5c-4f57-8cf1-3766a621dd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:42.577800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90011f8a-3e5c-4f57-8cf1-3766a621dd03 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.578149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90011f8a-3e5c-4f57-8cf1-3766a621dd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.579846 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90011f8a-3e5c-4f57-8cf1-3766a621dd03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.580956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90011f8a-3e5c-4f57-8cf1-3766a621dd03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:42.583468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90011f8a-3e5c-4f57-8cf1-3766a621dd03 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:42.641667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1003/2005] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:42 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:42.643350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df96623-4f00-432a-aaea-20816bcdff2f service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.656162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df96623-4f00-432a-aaea-20816bcdff2f service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:42.770132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92df73a-1637-4fb5-ba2d-961c1b94d3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:42.772258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92df73a-1637-4fb5-ba2d-961c1b94d3cf admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.772818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92df73a-1637-4fb5-ba2d-961c1b94d3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.773343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92df73a-1637-4fb5-ba2d-961c1b94d3cf admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.775253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92df73a-1637-4fb5-ba2d-961c1b94d3cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:42.777988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92df73a-1637-4fb5-ba2d-961c1b94d3cf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:42.812318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1003/2006] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:42 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:42.816118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90011f8a-3e5c-4f57-8cf1-3766a621dd03 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.836020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90011f8a-3e5c-4f57-8cf1-3766a621dd03 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:42.842535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af470b-6bdf-4c30-945d-ae08eb196b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:42.843496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af470b-6bdf-4c30-945d-ae08eb196b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.843925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af470b-6bdf-4c30-945d-ae08eb196b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.844644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af470b-6bdf-4c30-945d-ae08eb196b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.845679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af470b-6bdf-4c30-945d-ae08eb196b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:42.847638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af470b-6bdf-4c30-945d-ae08eb196b65 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:42.859027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1004/2007] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:42.863381 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92df73a-1637-4fb5-ba2d-961c1b94d3cf admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.875713 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92df73a-1637-4fb5-ba2d-961c1b94d3cf admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:42.883170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a6d9-71be-435d-b911-8dbd38a5b1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:42.883324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a6d9-71be-435d-b911-8dbd38a5b1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.884115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a6d9-71be-435d-b911-8dbd38a5b1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.884693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a6d9-71be-435d-b911-8dbd38a5b1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.885458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003a6d9-71be-435d-b911-8dbd38a5b1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:42.888227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003a6d9-71be-435d-b911-8dbd38a5b1a8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:42.903294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1004/2008] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:42.908179 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af470b-6bdf-4c30-945d-ae08eb196b65 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.922918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af470b-6bdf-4c30-945d-ae08eb196b65 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:42.943972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9323c9-2f08-4bd8-bad0-adcedf704fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:42.945687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9323c9-2f08-4bd8-bad0-adcedf704fae admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:42.946926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9323c9-2f08-4bd8-bad0-adcedf704fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:42.947403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9323c9-2f08-4bd8-bad0-adcedf704fae admin admin] PATH_INFO: `/v3/projects/cdb863849c1b495fa402a9c090aa01cf/users/13754243f9a5490c83c3ddcda206ea65/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:42.967133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a6d9-71be-435d-b911-8dbd38a5b1a8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:42.969147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b003a6d9-71be-435d-b911-8dbd38a5b1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:42.983436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9323c9-2f08-4bd8-bad0-adcedf704fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdb863849c1b495fa402a9c090aa01cf, user_id=13754243f9a5490c83c3ddcda206ea65, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:42.989908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9323c9-2f08-4bd8-bad0-adcedf704fae admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:42.992930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a6d9-71be-435d-b911-8dbd38a5b1a8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:43.022861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:43.023805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:43.024764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:43.026165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] PATH_INFO: `/v3/projects/cdd1c47c02b246c1b3ad6f7dc6f153db` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:43.037976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1005/2009] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:42 2022] PUT /identity/v3/projects/cdb863849c1b495fa402a9c090aa01cf/users/13754243f9a5490c83c3ddcda206ea65/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:43.040501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdd1c47c02b246c1b3ad6f7dc6f153db)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:43.043986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:43.047521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea6bc3-8da6-4f0c-8319-76b294d770e8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:43.047827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea6bc3-8da6-4f0c-8319-76b294d770e8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:43.048653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea6bc3-8da6-4f0c-8319-76b294d770e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:43.092907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1bea6bc3-8da6-4f0c-8319-76b294d770e8 None None] MFA Rules not processed for user `13754243f9a5490c83c3ddcda206ea65`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:43.097041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1bea6bc3-8da6-4f0c-8319-76b294d770e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:43.132996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdd1c47c02b246c1b3ad6f7dc6f153db', 'request_id': 'req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.162595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdd1c47c02b246c1b3ad6f7dc6f153db', 'request_id': 'req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.170844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdd1c47c02b246c1b3ad6f7dc6f153db', 'request_id': 'req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.183186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1006/2010] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:43.188047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea6bc3-8da6-4f0c-8319-76b294d770e8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:43.188391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] The token cache is being invalidate because project cdd1c47c02b246c1b3ad6f7dc6f153db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:43.188835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.189694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.193005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:43.194127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1005/2011] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:42 2022] DELETE /identity/v3/projects/cdd1c47c02b246c1b3ad6f7dc6f153db => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:43.198436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:43.200624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:43.285314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:43.309029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e8e65-f5bd-4e55-8ddf-5093f7d23b82 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:43.314984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:43.315645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:43.315929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:43.316207 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] PATH_INFO: `/v3/users/80aaaf433f14489281d4fbbe9ad214c3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:43.317537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80aaaf433f14489281d4fbbe9ad214c3)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:43.319534 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:43.323928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea6bc3-8da6-4f0c-8319-76b294d770e8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:43.333188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c88fba-1ca8-45d7-865c-22bb27b9235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:43.334231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c88fba-1ca8-45d7-865c-22bb27b9235b admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:43.334722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c88fba-1ca8-45d7-865c-22bb27b9235b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:43.335297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c88fba-1ca8-45d7-865c-22bb27b9235b admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:43.336401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c88fba-1ca8-45d7-865c-22bb27b9235b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:43.339948 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c88fba-1ca8-45d7-865c-22bb27b9235b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:43.393919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1007/2012] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:43 2022] POST /identity/v3/projects => generated 367 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:43.394288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80aaaf433f14489281d4fbbe9ad214c3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.395341 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c88fba-1ca8-45d7-865c-22bb27b9235b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:43.397225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f3c88fba-1ca8-45d7-865c-22bb27b9235b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:43.437811 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80aaaf433f14489281d4fbbe9ad214c3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.438955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80aaaf433f14489281d4fbbe9ad214c3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.440967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80aaaf433f14489281d4fbbe9ad214c3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.462061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80aaaf433f14489281d4fbbe9ad214c3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:43.500643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1006/2013] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:43 2022] DELETE /identity/v3/users/80aaaf433f14489281d4fbbe9ad214c3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:43.551435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c88fba-1ca8-45d7-865c-22bb27b9235b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:43.671521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d99d9-42e6-419d-9fd8-f9970cdfd7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:43.672360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d99d9-42e6-419d-9fd8-f9970cdfd7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:43.672645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d99d9-42e6-419d-9fd8-f9970cdfd7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:43.672909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d99d9-42e6-419d-9fd8-f9970cdfd7f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:43.673545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d99d9-42e6-419d-9fd8-f9970cdfd7f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:43.675528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d99d9-42e6-419d-9fd8-f9970cdfd7f3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:43.704174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1008/2014] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:43 2022] POST /identity/v3/users => generated 290 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:43.707199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:43.709336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:43.725327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603e2d9-d280-4613-a25e-7590b62b8153 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:43.732849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11ea20-12fa-4bb4-96b5-0ec513306d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:43.733785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11ea20-12fa-4bb4-96b5-0ec513306d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:43.734287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11ea20-12fa-4bb4-96b5-0ec513306d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:43.735059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11ea20-12fa-4bb4-96b5-0ec513306d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:43.736035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11ea20-12fa-4bb4-96b5-0ec513306d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:43.738527 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11ea20-12fa-4bb4-96b5-0ec513306d72 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:43.752742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1007/2015] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:43.759975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d99d9-42e6-419d-9fd8-f9970cdfd7f3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:43.775633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d99d9-42e6-419d-9fd8-f9970cdfd7f3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:43.782487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee4623-bc0c-4d6f-9cb7-36d78612e680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:43.783583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4623-bc0c-4d6f-9cb7-36d78612e680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:43.784288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4623-bc0c-4d6f-9cb7-36d78612e680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:43.785251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4623-bc0c-4d6f-9cb7-36d78612e680 admin admin] PATH_INFO: `/v3/projects/2e278dfa600941ac9a01b3b41b4dc554/users/2b92ca9b7b994e24a5dbad819a9576c5/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:43.815211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee4623-bc0c-4d6f-9cb7-36d78612e680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e278dfa600941ac9a01b3b41b4dc554, user_id=2b92ca9b7b994e24a5dbad819a9576c5, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:43.819084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee4623-bc0c-4d6f-9cb7-36d78612e680 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:43.829472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:43.838922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1009/2016] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:43 2022] PUT /identity/v3/projects/2e278dfa600941ac9a01b3b41b4dc554/users/2b92ca9b7b994e24a5dbad819a9576c5/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:43.843074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11ea20-12fa-4bb4-96b5-0ec513306d72 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:43.861921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11ea20-12fa-4bb4-96b5-0ec513306d72 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:43.916311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee4623-bc0c-4d6f-9cb7-36d78612e680 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:43.918550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-05ee4623-bc0c-4d6f-9cb7-36d78612e680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:43.932156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc07929-9ad7-4ee7-b5b6-13efb7b76a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:43.932965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07929-9ad7-4ee7-b5b6-13efb7b76a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:43.933242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07929-9ad7-4ee7-b5b6-13efb7b76a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:43.933702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07929-9ad7-4ee7-b5b6-13efb7b76a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:43.934434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc07929-9ad7-4ee7-b5b6-13efb7b76a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:43.936650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc07929-9ad7-4ee7-b5b6-13efb7b76a60 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:43.948616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1008/2017] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:28:43.953794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc07929-9ad7-4ee7-b5b6-13efb7b76a60 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:43.966133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc07929-9ad7-4ee7-b5b6-13efb7b76a60 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:43.971089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b290b-4795-4cd4-a78c-3f9ceed122fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:43.971865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b290b-4795-4cd4-a78c-3f9ceed122fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:43.972151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b290b-4795-4cd4-a78c-3f9ceed122fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:43.972430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b290b-4795-4cd4-a78c-3f9ceed122fa admin admin] PATH_INFO: `/v3/projects/2e278dfa600941ac9a01b3b41b4dc554/users/2b92ca9b7b994e24a5dbad819a9576c5/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:43.975844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b290b-4795-4cd4-a78c-3f9ceed122fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e278dfa600941ac9a01b3b41b4dc554, user_id=2b92ca9b7b994e24a5dbad819a9576c5, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:43.980401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b290b-4795-4cd4-a78c-3f9ceed122fa admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.000651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1009/2018] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:43 2022] PUT /identity/v3/projects/2e278dfa600941ac9a01b3b41b4dc554/users/2b92ca9b7b994e24a5dbad819a9576c5/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:44.011856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821be12d-a29e-4242-a098-c6b1d6656e46 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.012490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821be12d-a29e-4242-a098-c6b1d6656e46 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.013099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821be12d-a29e-4242-a098-c6b1d6656e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.016207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee4623-bc0c-4d6f-9cb7-36d78612e680 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.052285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-821be12d-a29e-4242-a098-c6b1d6656e46 None None] MFA Rules not processed for user `2b92ca9b7b994e24a5dbad819a9576c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:44.053821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-821be12d-a29e-4242-a098-c6b1d6656e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:44.083068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.083898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.084287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.084710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] PATH_INFO: `/v3/projects/f40ebac4cdcd47b9a95c7a26f26ce5de` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.086019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40ebac4cdcd47b9a95c7a26f26ce5de)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.088652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.097385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:44.167634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1010/2019] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:44.179182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821be12d-a29e-4242-a098-c6b1d6656e46 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.180855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-821be12d-a29e-4242-a098-c6b1d6656e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:44.183830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40ebac4cdcd47b9a95c7a26f26ce5de', 'request_id': 'req-b078a3c6-6bde-4b89-bf64-9cf63616a81a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:44.192448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40ebac4cdcd47b9a95c7a26f26ce5de', 'request_id': 'req-b078a3c6-6bde-4b89-bf64-9cf63616a81a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:44.202574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40ebac4cdcd47b9a95c7a26f26ce5de', 'request_id': 'req-b078a3c6-6bde-4b89-bf64-9cf63616a81a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:44.230170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] The token cache is being invalidate because project f40ebac4cdcd47b9a95c7a26f26ce5de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:44.230170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:44.230170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b078a3c6-6bde-4b89-bf64-9cf63616a81a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:44.230170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1010/2020] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:43 2022] DELETE /identity/v3/projects/f40ebac4cdcd47b9a95c7a26f26ce5de => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:44.230170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454734a-7508-42d0-9017-3e7300ceb251 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.230690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454734a-7508-42d0-9017-3e7300ceb251 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.230690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454734a-7508-42d0-9017-3e7300ceb251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.253825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4454734a-7508-42d0-9017-3e7300ceb251 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:44.255105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4454734a-7508-42d0-9017-3e7300ceb251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:44.259355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821be12d-a29e-4242-a098-c6b1d6656e46 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.298239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:44.323366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.324426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.324631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.325001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.327185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:44.334237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1011/2021] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:44.340237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de68e6-218d-4eaa-be02-df069a890e3c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.340511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de68e6-218d-4eaa-be02-df069a890e3c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.340793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de68e6-218d-4eaa-be02-df069a890e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.362823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-21de68e6-218d-4eaa-be02-df069a890e3c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:44.364040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-21de68e6-218d-4eaa-be02-df069a890e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:44.373297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1012/2022] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:44.377434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de68e6-218d-4eaa-be02-df069a890e3c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.388411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de68e6-218d-4eaa-be02-df069a890e3c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.393620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123c6a5-b4b7-4ef6-b5f8-266775a51d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.394340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123c6a5-b4b7-4ef6-b5f8-266775a51d5e admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.394693 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123c6a5-b4b7-4ef6-b5f8-266775a51d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.395039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123c6a5-b4b7-4ef6-b5f8-266775a51d5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.395751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123c6a5-b4b7-4ef6-b5f8-266775a51d5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:44.396127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123c6a5-b4b7-4ef6-b5f8-266775a51d5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.397843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123c6a5-b4b7-4ef6-b5f8-266775a51d5e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.402682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.404416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.411826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1013/2023] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:28:44.418170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1011/2024] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:44.418402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123c6a5-b4b7-4ef6-b5f8-266775a51d5e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.428017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123c6a5-b4b7-4ef6-b5f8-266775a51d5e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.432833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa90f87-920c-480e-bdaf-4ff897d162a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.433431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa90f87-920c-480e-bdaf-4ff897d162a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.433712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa90f87-920c-480e-bdaf-4ff897d162a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.434019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa90f87-920c-480e-bdaf-4ff897d162a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.434653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa90f87-920c-480e-bdaf-4ff897d162a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.436540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa90f87-920c-480e-bdaf-4ff897d162a8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.466774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.468469 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:44.472696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1014/2025] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:44 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:44.473636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa90f87-920c-480e-bdaf-4ff897d162a8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.482449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0fdde-e55f-4b37-9684-768f9e775515 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.514920 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:44.535236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f8d2a-0ab8-402b-8257-4ebdf2e9913c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.536395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f8d2a-0ab8-402b-8257-4ebdf2e9913c service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.536675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f8d2a-0ab8-402b-8257-4ebdf2e9913c service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.536956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f8d2a-0ab8-402b-8257-4ebdf2e9913c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.538843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-993f8d2a-0ab8-402b-8257-4ebdf2e9913c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:44.539437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa90f87-920c-480e-bdaf-4ff897d162a8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.544261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27e73f-c530-4cea-a7fe-ca8f4f86d949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.544878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27e73f-c530-4cea-a7fe-ca8f4f86d949 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.545159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27e73f-c530-4cea-a7fe-ca8f4f86d949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.545430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27e73f-c530-4cea-a7fe-ca8f4f86d949 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.546092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27e73f-c530-4cea-a7fe-ca8f4f86d949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.548048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27e73f-c530-4cea-a7fe-ca8f4f86d949 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.578227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1015/2026] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:44 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:44.579364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27e73f-c530-4cea-a7fe-ca8f4f86d949 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.591381 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27e73f-c530-4cea-a7fe-ca8f4f86d949 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.609377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a45e3-7fe1-485b-bc84-f87b5ec2a7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.610192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a45e3-7fe1-485b-bc84-f87b5ec2a7bd admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.613744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a45e3-7fe1-485b-bc84-f87b5ec2a7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.613744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a45e3-7fe1-485b-bc84-f87b5ec2a7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.613744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a45e3-7fe1-485b-bc84-f87b5ec2a7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.613744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a45e3-7fe1-485b-bc84-f87b5ec2a7bd admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.624716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f8d2a-0ab8-402b-8257-4ebdf2e9913c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.626258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f8d2a-0ab8-402b-8257-4ebdf2e9913c service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.628090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1016/2027] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:44.632509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a45e3-7fe1-485b-bc84-f87b5ec2a7bd admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.637624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1012/2028] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:44.645172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a45e3-7fe1-485b-bc84-f87b5ec2a7bd admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.650357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bf121-b135-471a-8096-09c9127deca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.651060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bf121-b135-471a-8096-09c9127deca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.651287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bf121-b135-471a-8096-09c9127deca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.651552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bf121-b135-471a-8096-09c9127deca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.652193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bf121-b135-471a-8096-09c9127deca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.653792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bf121-b135-471a-8096-09c9127deca3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.665205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1017/2029] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:44.669283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f8d2a-0ab8-402b-8257-4ebdf2e9913c service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.680139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f8d2a-0ab8-402b-8257-4ebdf2e9913c service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.685528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dac425-a9cd-4dcb-adba-26720b8997c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.686229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dac425-a9cd-4dcb-adba-26720b8997c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.686651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dac425-a9cd-4dcb-adba-26720b8997c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.686930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dac425-a9cd-4dcb-adba-26720b8997c5 admin admin] PATH_INFO: `/v3/projects/e7fd85826edf485b9c928bd810b14a97/users/db7af10fc0ee4ce29ad0785fe071015d/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.703123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dac425-a9cd-4dcb-adba-26720b8997c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7fd85826edf485b9c928bd810b14a97, user_id=db7af10fc0ee4ce29ad0785fe071015d, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.705760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dac425-a9cd-4dcb-adba-26720b8997c5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:44.724918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1013/2030] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:44 2022] PUT /identity/v3/projects/e7fd85826edf485b9c928bd810b14a97/users/db7af10fc0ee4ce29ad0785fe071015d/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:44.736013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde00f9-2616-4c77-b21b-67a8cdf32bc0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.736296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde00f9-2616-4c77-b21b-67a8cdf32bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.736835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde00f9-2616-4c77-b21b-67a8cdf32bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.773192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fcde00f9-2616-4c77-b21b-67a8cdf32bc0 None None] MFA Rules not processed for user `db7af10fc0ee4ce29ad0785fe071015d`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:44.774415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fcde00f9-2616-4c77-b21b-67a8cdf32bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:44.834549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dac425-a9cd-4dcb-adba-26720b8997c5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.837115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a4dac425-a9cd-4dcb-adba-26720b8997c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:44.876820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:44.899049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1018/2031] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:44.900738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde00f9-2616-4c77-b21b-67a8cdf32bc0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:44.943513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dac425-a9cd-4dcb-adba-26720b8997c5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:44.950106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:44.950873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:44.951250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:44.951744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] PATH_INFO: `/v3/users/ea890d42ec1e4b0682ee7a19a016b130` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:44.952995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea890d42ec1e4b0682ee7a19a016b130)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:44.955197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.020257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea890d42ec1e4b0682ee7a19a016b130'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.032093 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea890d42ec1e4b0682ee7a19a016b130'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.033026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea890d42ec1e4b0682ee7a19a016b130'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.034657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea890d42ec1e4b0682ee7a19a016b130'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.049814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde00f9-2616-4c77-b21b-67a8cdf32bc0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.050118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53feee16-46ad-4a52-95d4-daa904188f8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea890d42ec1e4b0682ee7a19a016b130'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.050118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:45.054835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db2d00-e053-4a95-b5e6-aaf66470fe8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.056218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db2d00-e053-4a95-b5e6-aaf66470fe8a admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.056566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db2d00-e053-4a95-b5e6-aaf66470fe8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.057214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db2d00-e053-4a95-b5e6-aaf66470fe8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.057947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db2d00-e053-4a95-b5e6-aaf66470fe8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.060330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db2d00-e053-4a95-b5e6-aaf66470fe8a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.080378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1014/2032] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:44 2022] DELETE /identity/v3/users/ea890d42ec1e4b0682ee7a19a016b130 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:45.086232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73435f-0b4a-4e3a-9214-cbd69c8b1ca6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.087757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73435f-0b4a-4e3a-9214-cbd69c8b1ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.087757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73435f-0b4a-4e3a-9214-cbd69c8b1ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.125970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1019/2033] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:44 2022] POST /identity/v3/projects => generated 379 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:45.126163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4e73435f-0b4a-4e3a-9214-cbd69c8b1ca6 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:45.126733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4e73435f-0b4a-4e3a-9214-cbd69c8b1ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:45.129827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db2d00-e053-4a95-b5e6-aaf66470fe8a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.132419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c3db2d00-e053-4a95-b5e6-aaf66470fe8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:45.203064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1015/2034] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:45.213360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73435f-0b4a-4e3a-9214-cbd69c8b1ca6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.215598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4e73435f-0b4a-4e3a-9214-cbd69c8b1ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:45.218168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db2d00-e053-4a95-b5e6-aaf66470fe8a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.223760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062820e0-def4-47c7-8e3a-ffc45fa54897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.224441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062820e0-def4-47c7-8e3a-ffc45fa54897 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.224963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062820e0-def4-47c7-8e3a-ffc45fa54897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.229721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062820e0-def4-47c7-8e3a-ffc45fa54897 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.229721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062820e0-def4-47c7-8e3a-ffc45fa54897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.230489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062820e0-def4-47c7-8e3a-ffc45fa54897 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.265132 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1020/2035] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:45 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:45.266674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062820e0-def4-47c7-8e3a-ffc45fa54897 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.282260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062820e0-def4-47c7-8e3a-ffc45fa54897 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.300143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f11f5-4539-46f7-bc35-8b6e0f7082dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.301053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f11f5-4539-46f7-bc35-8b6e0f7082dd admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.301974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f11f5-4539-46f7-bc35-8b6e0f7082dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.302715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f11f5-4539-46f7-bc35-8b6e0f7082dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.304419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f11f5-4539-46f7-bc35-8b6e0f7082dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.306452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f11f5-4539-46f7-bc35-8b6e0f7082dd admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.316939 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73435f-0b4a-4e3a-9214-cbd69c8b1ca6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.318260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1021/2036] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:28:45.322426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f11f5-4539-46f7-bc35-8b6e0f7082dd admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.323354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a46d07-5a1c-4ad6-bb73-195d8f762d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.323975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a46d07-5a1c-4ad6-bb73-195d8f762d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.324249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a46d07-5a1c-4ad6-bb73-195d8f762d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.324517 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a46d07-5a1c-4ad6-bb73-195d8f762d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.334989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a46d07-5a1c-4ad6-bb73-195d8f762d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.335148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f11f5-4539-46f7-bc35-8b6e0f7082dd admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.337155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a46d07-5a1c-4ad6-bb73-195d8f762d22 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.340364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5b1a4-6ed4-45c7-8d0e-7a84384f0c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.340971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5b1a4-6ed4-45c7-8d0e-7a84384f0c7b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.341253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5b1a4-6ed4-45c7-8d0e-7a84384f0c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.341527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5b1a4-6ed4-45c7-8d0e-7a84384f0c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.342190 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5b1a4-6ed4-45c7-8d0e-7a84384f0c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.343898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5b1a4-6ed4-45c7-8d0e-7a84384f0c7b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.356223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1022/2037] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:45.358116 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1016/2038] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:45.362919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5b1a4-6ed4-45c7-8d0e-7a84384f0c7b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.375767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5b1a4-6ed4-45c7-8d0e-7a84384f0c7b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.382251 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82c5cf-5099-47b3-a7d8-0b0256efc1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.382926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82c5cf-5099-47b3-a7d8-0b0256efc1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.383208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82c5cf-5099-47b3-a7d8-0b0256efc1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.383508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82c5cf-5099-47b3-a7d8-0b0256efc1ea admin admin] PATH_INFO: `/v3/projects/83e92af2602e4c6dadd6adce44e09aba/users/cab3d6ffd9464f708fa14c55d3fcc937/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.411275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82c5cf-5099-47b3-a7d8-0b0256efc1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=83e92af2602e4c6dadd6adce44e09aba, user_id=cab3d6ffd9464f708fa14c55d3fcc937, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.414541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82c5cf-5099-47b3-a7d8-0b0256efc1ea admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.433327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1023/2039] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:45 2022] PUT /identity/v3/projects/83e92af2602e4c6dadd6adce44e09aba/users/cab3d6ffd9464f708fa14c55d3fcc937/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:45.447838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f3106-9449-4c2a-8f83-d4154bf8b993 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.448564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f3106-9449-4c2a-8f83-d4154bf8b993 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.448846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f3106-9449-4c2a-8f83-d4154bf8b993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.479087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-692f3106-9449-4c2a-8f83-d4154bf8b993 None None] MFA Rules not processed for user `cab3d6ffd9464f708fa14c55d3fcc937`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:45.480328 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-692f3106-9449-4c2a-8f83-d4154bf8b993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:45.561159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1017/2040] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:45.565297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82c5cf-5099-47b3-a7d8-0b0256efc1ea admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.598019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:45.643426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82c5cf-5099-47b3-a7d8-0b0256efc1ea admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.650565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b479523-d89d-451c-936d-8073e74fe24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.651315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b479523-d89d-451c-936d-8073e74fe24a admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.651604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b479523-d89d-451c-936d-8073e74fe24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.651883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b479523-d89d-451c-936d-8073e74fe24a admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.652581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b479523-d89d-451c-936d-8073e74fe24a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.654596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b479523-d89d-451c-936d-8073e74fe24a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.660806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f3106-9449-4c2a-8f83-d4154bf8b993 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.662252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-692f3106-9449-4c2a-8f83-d4154bf8b993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:45.676578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f3106-9449-4c2a-8f83-d4154bf8b993 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.681605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.682363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.682738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.683050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] PATH_INFO: `/v3/projects/ba8fd917635c4153b0e7d73c38e687fa` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.684731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba8fd917635c4153b0e7d73c38e687fa)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.686973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.690769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1024/2041] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:45 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:45.694978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b479523-d89d-451c-936d-8073e74fe24a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.700378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:45.749053 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba8fd917635c4153b0e7d73c38e687fa', 'request_id': 'req-6397f5ab-0370-4624-85c2-12fe685cff17'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.757443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba8fd917635c4153b0e7d73c38e687fa', 'request_id': 'req-6397f5ab-0370-4624-85c2-12fe685cff17'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.764433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba8fd917635c4153b0e7d73c38e687fa', 'request_id': 'req-6397f5ab-0370-4624-85c2-12fe685cff17'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.764885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b479523-d89d-451c-936d-8073e74fe24a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.777320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] The token cache is being invalidate because project ba8fd917635c4153b0e7d73c38e687fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:45.777615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.778324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6397f5ab-0370-4624-85c2-12fe685cff17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:45.779831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1018/2042] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:45 2022] DELETE /identity/v3/projects/ba8fd917635c4153b0e7d73c38e687fa => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:45.784482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca64eb-9d06-4a15-84d6-eeaf2bc97449 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.784711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca64eb-9d06-4a15-84d6-eeaf2bc97449 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.784991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca64eb-9d06-4a15-84d6-eeaf2bc97449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.805632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-32ca64eb-9d06-4a15-84d6-eeaf2bc97449 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:45.806914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-32ca64eb-9d06-4a15-84d6-eeaf2bc97449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:45.829177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8a3b7-59f0-47a7-8083-3d39fa6ee78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.829788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8a3b7-59f0-47a7-8083-3d39fa6ee78c admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.830212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8a3b7-59f0-47a7-8083-3d39fa6ee78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.830497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8a3b7-59f0-47a7-8083-3d39fa6ee78c admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.831132 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8a3b7-59f0-47a7-8083-3d39fa6ee78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.833064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8a3b7-59f0-47a7-8083-3d39fa6ee78c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.857259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1025/2043] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:45 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:45.861041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8a3b7-59f0-47a7-8083-3d39fa6ee78c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.863931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ebe8a3b7-59f0-47a7-8083-3d39fa6ee78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:45.870525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1019/2044] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:45.875941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8a3b7-59f0-47a7-8083-3d39fa6ee78c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.878880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8342f4-3911-4a7d-9ea1-a31e8c42948d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.879176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8342f4-3911-4a7d-9ea1-a31e8c42948d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.879451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8342f4-3911-4a7d-9ea1-a31e8c42948d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.880932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ba9d1-de5b-45fc-b793-c628ce576708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.881528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba9d1-de5b-45fc-b793-c628ce576708 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.881808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba9d1-de5b-45fc-b793-c628ce576708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.882119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba9d1-de5b-45fc-b793-c628ce576708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.882780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ba9d1-de5b-45fc-b793-c628ce576708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.884893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ba9d1-de5b-45fc-b793-c628ce576708 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.895159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1026/2045] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:28:45.900059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ba9d1-de5b-45fc-b793-c628ce576708 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.904010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5d8342f4-3911-4a7d-9ea1-a31e8c42948d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:45.905221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5d8342f4-3911-4a7d-9ea1-a31e8c42948d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:45.910000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ba9d1-de5b-45fc-b793-c628ce576708 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.913261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1020/2046] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:45.916548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6594a0-0dfa-4bb1-8da3-a5acc542a446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.916548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6594a0-0dfa-4bb1-8da3-a5acc542a446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.916548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6594a0-0dfa-4bb1-8da3-a5acc542a446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.916548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6594a0-0dfa-4bb1-8da3-a5acc542a446 admin admin] PATH_INFO: `/v3/projects/538670da7cd4440ebf6446c375a9f103/users/4b98949b370a41f1abd9c60362098606/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.917345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8342f4-3911-4a7d-9ea1-a31e8c42948d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.928037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8342f4-3911-4a7d-9ea1-a31e8c42948d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:45.931162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6594a0-0dfa-4bb1-8da3-a5acc542a446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538670da7cd4440ebf6446c375a9f103, user_id=4b98949b370a41f1abd9c60362098606, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.932863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daec09-59f6-4d48-8053-dee4b311fe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:45.933474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daec09-59f6-4d48-8053-dee4b311fe32 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:45.933754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daec09-59f6-4d48-8053-dee4b311fe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:45.933857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6594a0-0dfa-4bb1-8da3-a5acc542a446 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.934060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daec09-59f6-4d48-8053-dee4b311fe32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:45.934701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daec09-59f6-4d48-8053-dee4b311fe32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:45.935005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daec09-59f6-4d48-8053-dee4b311fe32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:45.936584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daec09-59f6-4d48-8053-dee4b311fe32 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:45.948845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1027/2047] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:45 2022] PUT /identity/v3/projects/538670da7cd4440ebf6446c375a9f103/users/4b98949b370a41f1abd9c60362098606/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:45.953278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1021/2048] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:28:45.953408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6594a0-0dfa-4bb1-8da3-a5acc542a446 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.954221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daec09-59f6-4d48-8053-dee4b311fe32 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:45.964070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:45.966386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daec09-59f6-4d48-8053-dee4b311fe32 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.025506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6594a0-0dfa-4bb1-8da3-a5acc542a446 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.028475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3519cc-7d32-42ae-bf9e-66747e46fadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.029127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3519cc-7d32-42ae-bf9e-66747e46fadf admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.029414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3519cc-7d32-42ae-bf9e-66747e46fadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.029692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3519cc-7d32-42ae-bf9e-66747e46fadf admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.030437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f15593-a9b4-4343-92fc-f11f9f2fd62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.030563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3519cc-7d32-42ae-bf9e-66747e46fadf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.031190 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15593-a9b4-4343-92fc-f11f9f2fd62f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.031472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15593-a9b4-4343-92fc-f11f9f2fd62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.031745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15593-a9b4-4343-92fc-f11f9f2fd62f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.032389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f15593-a9b4-4343-92fc-f11f9f2fd62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.032491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3519cc-7d32-42ae-bf9e-66747e46fadf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.034045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f15593-a9b4-4343-92fc-f11f9f2fd62f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.044815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1028/2049] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:46.048590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f15593-a9b4-4343-92fc-f11f9f2fd62f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:46.060472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f15593-a9b4-4343-92fc-f11f9f2fd62f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.067494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1022/2050] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:45 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:46.068591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0529e4-8e3c-439e-a124-ac209e58da42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.069270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0529e4-8e3c-439e-a124-ac209e58da42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.069544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0529e4-8e3c-439e-a124-ac209e58da42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.069813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0529e4-8e3c-439e-a124-ac209e58da42 admin admin] PATH_INFO: `/v3/projects/538670da7cd4440ebf6446c375a9f103/users/4b98949b370a41f1abd9c60362098606/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.072659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3519cc-7d32-42ae-bf9e-66747e46fadf admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:46.073656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0529e4-8e3c-439e-a124-ac209e58da42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538670da7cd4440ebf6446c375a9f103, user_id=4b98949b370a41f1abd9c60362098606, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.076294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0529e4-8e3c-439e-a124-ac209e58da42 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.078947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:46.088375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1029/2051] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:46 2022] PUT /identity/v3/projects/538670da7cd4440ebf6446c375a9f103/users/4b98949b370a41f1abd9c60362098606/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:46.102631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7289f-1410-452a-bde9-29c1d3f6b907 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.102916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7289f-1410-452a-bde9-29c1d3f6b907 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.103189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7289f-1410-452a-bde9-29c1d3f6b907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.136020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e6e7289f-1410-452a-bde9-29c1d3f6b907 None None] MFA Rules not processed for user `4b98949b370a41f1abd9c60362098606`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:46.137285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e6e7289f-1410-452a-bde9-29c1d3f6b907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:46.155009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3519cc-7d32-42ae-bf9e-66747e46fadf admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.219041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3207db-b3a4-4656-910c-f92f9fec3d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.219785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3207db-b3a4-4656-910c-f92f9fec3d63 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.220168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3207db-b3a4-4656-910c-f92f9fec3d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.220598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3207db-b3a4-4656-910c-f92f9fec3d63 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.221306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3207db-b3a4-4656-910c-f92f9fec3d63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.223963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3207db-b3a4-4656-910c-f92f9fec3d63 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.228977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1030/2052] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:46.241438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7289f-1410-452a-bde9-29c1d3f6b907 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:46.244742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e6e7289f-1410-452a-bde9-29c1d3f6b907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:46.265373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1023/2053] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:46 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:46.267061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3207db-b3a4-4656-910c-f92f9fec3d63 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:46.280090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3207db-b3a4-4656-910c-f92f9fec3d63 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.285997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175e802-9d80-48b3-8b60-8c6422f331f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.286692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175e802-9d80-48b3-8b60-8c6422f331f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.287038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175e802-9d80-48b3-8b60-8c6422f331f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.287378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175e802-9d80-48b3-8b60-8c6422f331f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.288083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175e802-9d80-48b3-8b60-8c6422f331f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.289909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175e802-9d80-48b3-8b60-8c6422f331f9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.303890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1024/2054] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:46.308618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175e802-9d80-48b3-8b60-8c6422f331f9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:46.310302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7289f-1410-452a-bde9-29c1d3f6b907 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.319190 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5dc3a-997b-4f7c-9a8b-6182ad217364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.319457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175e802-9d80-48b3-8b60-8c6422f331f9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.320648 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5dc3a-997b-4f7c-9a8b-6182ad217364 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.320931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5dc3a-997b-4f7c-9a8b-6182ad217364 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.321779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5dc3a-997b-4f7c-9a8b-6182ad217364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.325090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee647122-7493-40ef-900f-e2461e557c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.325687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee647122-7493-40ef-900f-e2461e557c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.325996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee647122-7493-40ef-900f-e2461e557c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.326274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee647122-7493-40ef-900f-e2461e557c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.327043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee647122-7493-40ef-900f-e2461e557c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.328649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee647122-7493-40ef-900f-e2461e557c61 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.331435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5dc3a-997b-4f7c-9a8b-6182ad217364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.332876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5dc3a-997b-4f7c-9a8b-6182ad217364 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.341669 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1025/2055] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:28:46.345680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee647122-7493-40ef-900f-e2461e557c61 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:46.346499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1031/2056] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:46.359775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee647122-7493-40ef-900f-e2461e557c61 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.364884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3db95a-81d8-4536-982d-6b1ab9961193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.365568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db95a-81d8-4536-982d-6b1ab9961193 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.365845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db95a-81d8-4536-982d-6b1ab9961193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.366201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db95a-81d8-4536-982d-6b1ab9961193 admin admin] PATH_INFO: `/v3/projects/ab6e361f9f314871b1593d2ee08591e8/users/9e180c25dabb4c188ba1a3bc4f4ce365/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.399418 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3db95a-81d8-4536-982d-6b1ab9961193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6e361f9f314871b1593d2ee08591e8, user_id=9e180c25dabb4c188ba1a3bc4f4ce365, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.402232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3db95a-81d8-4536-982d-6b1ab9961193 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.418967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1026/2057] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:46 2022] PUT /identity/v3/projects/ab6e361f9f314871b1593d2ee08591e8/users/9e180c25dabb4c188ba1a3bc4f4ce365/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:46.420566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5dc3a-997b-4f7c-9a8b-6182ad217364 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:46.432445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6fe8c-7d1a-478a-8d90-9f6a1bda7b9e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.433054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6fe8c-7d1a-478a-8d90-9f6a1bda7b9e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.434521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6fe8c-7d1a-478a-8d90-9f6a1bda7b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.436066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5dc3a-997b-4f7c-9a8b-6182ad217364 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.463952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:46.514552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-39c6fe8c-7d1a-478a-8d90-9f6a1bda7b9e None None] MFA Rules not processed for user `9e180c25dabb4c188ba1a3bc4f4ce365`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:46.515886 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-39c6fe8c-7d1a-478a-8d90-9f6a1bda7b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:46.522126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.523059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.523523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.524014 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.527451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:46.616243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:46.658107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1027/2058] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:46.697665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6fe8c-7d1a-478a-8d90-9f6a1bda7b9e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:46.699309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-39c6fe8c-7d1a-478a-8d90-9f6a1bda7b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:46.701867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.711200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.721640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6fe8c-7d1a-478a-8d90-9f6a1bda7b9e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:46.722115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1032/2059] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:46.734364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15068c1-d9d7-45e6-8f67-012799cfc646 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:46.735026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15068c1-d9d7-45e6-8f67-012799cfc646 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:46.735305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15068c1-d9d7-45e6-8f67-012799cfc646 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:46.735576 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15068c1-d9d7-45e6-8f67-012799cfc646 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:46.759595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15068c1-d9d7-45e6-8f67-012799cfc646 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:46.759595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15068c1-d9d7-45e6-8f67-012799cfc646 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:46.772659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1028/2060] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:48.479012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:48.482038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:48.569709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823961a-3915-49d0-a276-2337ec8d6d1b service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:48.575775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:48.575933 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:48.576205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:48.576484 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] PATH_INFO: `/v3/users/13754243f9a5490c83c3ddcda206ea65` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:48.577654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13754243f9a5490c83c3ddcda206ea65)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:48.579636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:48.661035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13754243f9a5490c83c3ddcda206ea65'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:48.662255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13754243f9a5490c83c3ddcda206ea65'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:48.685205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13754243f9a5490c83c3ddcda206ea65'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:48.686351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13754243f9a5490c83c3ddcda206ea65'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:48.697712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13754243f9a5490c83c3ddcda206ea65'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:48.722503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:48.746908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1033/2061] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:48 2022] DELETE /identity/v3/users/13754243f9a5490c83c3ddcda206ea65 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:48.748955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7ca23-cb8b-4427-8414-3da1657f4d62 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:48.750251 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7ca23-cb8b-4427-8414-3da1657f4d62 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:48.750651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7ca23-cb8b-4427-8414-3da1657f4d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:48.792506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-45e7ca23-cb8b-4427-8414-3da1657f4d62 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:48.794900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-45e7ca23-cb8b-4427-8414-3da1657f4d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:48.907872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1029/2062] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:48.915002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:48.919622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:49.003112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a80295-b0d7-4462-9534-7787da7373c7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:49.008284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4636fb5-d89b-4729-9cbf-d2b7d00df372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:49.009544 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4636fb5-d89b-4729-9cbf-d2b7d00df372 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:49.009824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4636fb5-d89b-4729-9cbf-d2b7d00df372 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:49.010960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4636fb5-d89b-4729-9cbf-d2b7d00df372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:49.022833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4636fb5-d89b-4729-9cbf-d2b7d00df372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:49.024869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4636fb5-d89b-4729-9cbf-d2b7d00df372 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:49.037167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1034/2063] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:49.375464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7ca23-cb8b-4427-8414-3da1657f4d62 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:49.376883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-45e7ca23-cb8b-4427-8414-3da1657f4d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:49.398373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7ca23-cb8b-4427-8414-3da1657f4d62 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:49.406902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:49.408260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:49.409760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:49.410572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] PATH_INFO: `/v3/projects/cdb863849c1b495fa402a9c090aa01cf` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:49.412426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdb863849c1b495fa402a9c090aa01cf)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:49.415789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:49.468302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdb863849c1b495fa402a9c090aa01cf', 'request_id': 'req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.476712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdb863849c1b495fa402a9c090aa01cf', 'request_id': 'req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.483513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdb863849c1b495fa402a9c090aa01cf', 'request_id': 'req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.496123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] The token cache is being invalidate because project cdb863849c1b495fa402a9c090aa01cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:49.496425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.497082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.498612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1030/2064] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:49 2022] DELETE /identity/v3/projects/cdb863849c1b495fa402a9c090aa01cf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:49.502764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4636fb5-d89b-4729-9cbf-d2b7d00df372 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:49.515075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4636fb5-d89b-4729-9cbf-d2b7d00df372 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:49.532640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:49.534184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:49.580780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:49.581513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:49.581996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:49.582259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] PATH_INFO: `/v3/users/2b92ca9b7b994e24a5dbad819a9576c5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:49.583399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b92ca9b7b994e24a5dbad819a9576c5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:49.597763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:49.619582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:49.631425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd2d9b-4536-448d-91f8-fb4dfcf1cf6e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:49.638965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:49.639881 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:49.640374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:49.640981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] PATH_INFO: `/v3/users/9e180c25dabb4c188ba1a3bc4f4ce365` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:49.642381 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e180c25dabb4c188ba1a3bc4f4ce365)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:49.645478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:49.669619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b92ca9b7b994e24a5dbad819a9576c5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.671622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b92ca9b7b994e24a5dbad819a9576c5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.685835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b92ca9b7b994e24a5dbad819a9576c5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.686785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b92ca9b7b994e24a5dbad819a9576c5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.705989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e09053fc-aa60-4077-8cde-8a23c9aa6dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b92ca9b7b994e24a5dbad819a9576c5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.728566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e180c25dabb4c188ba1a3bc4f4ce365'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.737330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1035/2065] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:49 2022] DELETE /identity/v3/users/2b92ca9b7b994e24a5dbad819a9576c5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:49.739654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e180c25dabb4c188ba1a3bc4f4ce365'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.740397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e180c25dabb4c188ba1a3bc4f4ce365'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.741211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e180c25dabb4c188ba1a3bc4f4ce365'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.750406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e180c25dabb4c188ba1a3bc4f4ce365'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:49.778038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1031/2066] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:49 2022] DELETE /identity/v3/users/9e180c25dabb4c188ba1a3bc4f4ce365 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:49.782911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a0ddd-e07d-44fb-bbd5-c0e19189ae28 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:49.783248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a0ddd-e07d-44fb-bbd5-c0e19189ae28 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:49.783488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a0ddd-e07d-44fb-bbd5-c0e19189ae28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:49.821323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b17a0ddd-e07d-44fb-bbd5-c0e19189ae28 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:49.822598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b17a0ddd-e07d-44fb-bbd5-c0e19189ae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:49.845871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:49.931966 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1036/2067] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:49.974557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:49.976565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:50.064739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a0ddd-e07d-44fb-bbd5-c0e19189ae28 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:50.066797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b17a0ddd-e07d-44fb-bbd5-c0e19189ae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:50.083142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a0ddd-e07d-44fb-bbd5-c0e19189ae28 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:50.109996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:50.109996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:50.109996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:50.109996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] PATH_INFO: `/v3/users/db7af10fc0ee4ce29ad0785fe071015d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:50.110445 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7af10fc0ee4ce29ad0785fe071015d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:50.111696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622db810-6e66-41c8-97bd-6bb7f188ca50 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:50.112201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:50.116649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e65209-1798-44de-b7a6-18a9bc6465fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:50.117352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e65209-1798-44de-b7a6-18a9bc6465fb service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:50.117880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e65209-1798-44de-b7a6-18a9bc6465fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:50.118319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e65209-1798-44de-b7a6-18a9bc6465fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:50.130662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e65209-1798-44de-b7a6-18a9bc6465fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:50.132229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e65209-1798-44de-b7a6-18a9bc6465fb service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:50.153197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1032/2068] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:50.210971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7af10fc0ee4ce29ad0785fe071015d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.212468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7af10fc0ee4ce29ad0785fe071015d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.229027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7af10fc0ee4ce29ad0785fe071015d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.230827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7af10fc0ee4ce29ad0785fe071015d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.246387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7d21ce06-7b7c-46b8-990c-e5acc63d0850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7af10fc0ee4ce29ad0785fe071015d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.270282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e65209-1798-44de-b7a6-18a9bc6465fb service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:50.272117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b8e65209-1798-44de-b7a6-18a9bc6465fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:50.294323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e65209-1798-44de-b7a6-18a9bc6465fb service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:50.304403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1037/2069] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:50 2022] DELETE /identity/v3/users/db7af10fc0ee4ce29ad0785fe071015d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:50.307037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cd50c-4e80-4fd9-b737-e02b9940d9a1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:50.307369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cd50c-4e80-4fd9-b737-e02b9940d9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:50.307648 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cd50c-4e80-4fd9-b737-e02b9940d9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:50.357877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-639cd50c-4e80-4fd9-b737-e02b9940d9a1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:50.359226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-639cd50c-4e80-4fd9-b737-e02b9940d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:50.451934 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:50.452603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:50.452883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:50.453338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] PATH_INFO: `/v3/projects/2e278dfa600941ac9a01b3b41b4dc554` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:50.455379 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e278dfa600941ac9a01b3b41b4dc554)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:50.460814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:50.518360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1038/2070] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:50.518938 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:50.546575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cd50c-4e80-4fd9-b737-e02b9940d9a1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:50.550327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-639cd50c-4e80-4fd9-b737-e02b9940d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:50.575385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e278dfa600941ac9a01b3b41b4dc554', 'request_id': 'req-14cdf995-f402-4b08-ba49-96ba12c8e3a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.592006 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e278dfa600941ac9a01b3b41b4dc554', 'request_id': 'req-14cdf995-f402-4b08-ba49-96ba12c8e3a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.622428 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e278dfa600941ac9a01b3b41b4dc554', 'request_id': 'req-14cdf995-f402-4b08-ba49-96ba12c8e3a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.667629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:50.692652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] The token cache is being invalidate because project 2e278dfa600941ac9a01b3b41b4dc554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:50.693617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.706665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-14cdf995-f402-4b08-ba49-96ba12c8e3a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:50.713909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1033/2071] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:50 2022] DELETE /identity/v3/projects/2e278dfa600941ac9a01b3b41b4dc554 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:50.725534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cd50c-4e80-4fd9-b737-e02b9940d9a1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:50.726733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb15c-7acc-4ff1-a016-8b9a9a96b15b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:50.727250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb15c-7acc-4ff1-a016-8b9a9a96b15b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:50.727705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb15c-7acc-4ff1-a016-8b9a9a96b15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:50.777202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3cdcb15c-7acc-4ff1-a016-8b9a9a96b15b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:50.779390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3cdcb15c-7acc-4ff1-a016-8b9a9a96b15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:50.835153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07895c5-0d42-4b3d-92fd-0ce3b7e0dcd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:50.835911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07895c5-0d42-4b3d-92fd-0ce3b7e0dcd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:50.836326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07895c5-0d42-4b3d-92fd-0ce3b7e0dcd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:50.836717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07895c5-0d42-4b3d-92fd-0ce3b7e0dcd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:50.838630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b07895c5-0d42-4b3d-92fd-0ce3b7e0dcd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:50.905011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1034/2072] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:50.912371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcb15c-7acc-4ff1-a016-8b9a9a96b15b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:50.912371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3cdcb15c-7acc-4ff1-a016-8b9a9a96b15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:50.924867 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcb15c-7acc-4ff1-a016-8b9a9a96b15b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:50.929377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:50.933187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:50.933187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:50.933187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] PATH_INFO: `/v3/projects/ab6e361f9f314871b1593d2ee08591e8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:50.934972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6e361f9f314871b1593d2ee08591e8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:50.936229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07895c5-0d42-4b3d-92fd-0ce3b7e0dcd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:50.937249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:50.937681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07895c5-0d42-4b3d-92fd-0ce3b7e0dcd3 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:50.954690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1039/2073] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:50.987159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4e98-846d-4ba7-9e51-0bce6b8bb9f3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:50.987159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4e98-846d-4ba7-9e51-0bce6b8bb9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:50.987159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4e98-846d-4ba7-9e51-0bce6b8bb9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.003854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6c9a4e98-846d-4ba7-9e51-0bce6b8bb9f3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:51.005247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6c9a4e98-846d-4ba7-9e51-0bce6b8bb9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.035930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6e361f9f314871b1593d2ee08591e8', 'request_id': 'req-153581a5-9d6f-4ff2-9738-ee68dd104cc8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.048679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6e361f9f314871b1593d2ee08591e8', 'request_id': 'req-153581a5-9d6f-4ff2-9738-ee68dd104cc8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.057746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6e361f9f314871b1593d2ee08591e8', 'request_id': 'req-153581a5-9d6f-4ff2-9738-ee68dd104cc8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.079152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] The token cache is being invalidate because project ab6e361f9f314871b1593d2ee08591e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:51.079694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.080791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.086063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1035/2074] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:50 2022] DELETE /identity/v3/projects/ab6e361f9f314871b1593d2ee08591e8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:51.087497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.089055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.092758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1040/2075] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:51.096025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ada6e-680c-4860-8ea3-abdea7fd757f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.096025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ada6e-680c-4860-8ea3-abdea7fd757f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.096025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ada6e-680c-4860-8ea3-abdea7fd757f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.112898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153581a5-9d6f-4ff2-9738-ee68dd104cc8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.125920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d08ada6e-680c-4860-8ea3-abdea7fd757f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:51.126801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.127518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.127876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.128013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d08ada6e-680c-4860-8ea3-abdea7fd757f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.128274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] PATH_INFO: `/v3/users/75d4379ec87240389ce9c077886ab472` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.129501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d4379ec87240389ce9c077886ab472)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.131516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.138355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1041/2076] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:51.139370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ada6e-680c-4860-8ea3-abdea7fd757f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.151253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ada6e-680c-4860-8ea3-abdea7fd757f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.158273 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb65c0b-9350-4669-b263-f142e4bdebf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.159843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb65c0b-9350-4669-b263-f142e4bdebf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.160968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb65c0b-9350-4669-b263-f142e4bdebf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.162010 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb65c0b-9350-4669-b263-f142e4bdebf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.162824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb65c0b-9350-4669-b263-f142e4bdebf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:51.163241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb65c0b-9350-4669-b263-f142e4bdebf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.165584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb65c0b-9350-4669-b263-f142e4bdebf4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.168481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:51.182284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1042/2077] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:28:51.182284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a58aa-c843-43ef-8abe-d926451b280b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.182284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a58aa-c843-43ef-8abe-d926451b280b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.182284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a58aa-c843-43ef-8abe-d926451b280b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.198747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d4379ec87240389ce9c077886ab472'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.208866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7b4a58aa-c843-43ef-8abe-d926451b280b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:51.210521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7b4a58aa-c843-43ef-8abe-d926451b280b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.210843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d4379ec87240389ce9c077886ab472'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.212040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d4379ec87240389ce9c077886ab472'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.213409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d4379ec87240389ce9c077886ab472'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.222285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d4379ec87240389ce9c077886ab472'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.228557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1043/2078] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:51.229462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a58aa-c843-43ef-8abe-d926451b280b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.230932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7b4a58aa-c843-43ef-8abe-d926451b280b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.246253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a58aa-c843-43ef-8abe-d926451b280b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.252224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3585b8-5567-4076-9933-50ad437fdb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.252837 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3585b8-5567-4076-9933-50ad437fdb65 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.253117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3585b8-5567-4076-9933-50ad437fdb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.253404 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3585b8-5567-4076-9933-50ad437fdb65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.254058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3585b8-5567-4076-9933-50ad437fdb65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.256471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3585b8-5567-4076-9933-50ad437fdb65 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.259926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1036/2079] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:51 2022] DELETE /identity/v3/users/75d4379ec87240389ce9c077886ab472 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:51.262890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.271560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:51.319475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1044/2080] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:51 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:51.325208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3585b8-5567-4076-9933-50ad437fdb65 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.359625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38ebc6-73a7-4930-b8b9-17e21105c1e1 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.435430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3585b8-5567-4076-9933-50ad437fdb65 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.443064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.443689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.444169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.444321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.444586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab57ae-b293-4eea-b208-8aa42aa24688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.444931 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:51.445225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.445607 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab57ae-b293-4eea-b208-8aa42aa24688 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.446081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab57ae-b293-4eea-b208-8aa42aa24688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.446510 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab57ae-b293-4eea-b208-8aa42aa24688 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.446882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.447401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab57ae-b293-4eea-b208-8aa42aa24688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.449886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab57ae-b293-4eea-b208-8aa42aa24688 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.460359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1037/2081] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:28:51.460959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.463637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.480843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6d3a4-f3b1-4f7f-8480-5058f7384aa0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.482351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1045/2082] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:51 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:51.483054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab57ae-b293-4eea-b208-8aa42aa24688 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.488592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.489349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.489755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.490179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] PATH_INFO: `/v3/projects/e7fd85826edf485b9c928bd810b14a97` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.492258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7fd85826edf485b9c928bd810b14a97)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.494577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.499064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab57ae-b293-4eea-b208-8aa42aa24688 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.505097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5af5f1-5619-48ec-a323-ffc404a53057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.505776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5af5f1-5619-48ec-a323-ffc404a53057 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.506738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5af5f1-5619-48ec-a323-ffc404a53057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.507013 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5af5f1-5619-48ec-a323-ffc404a53057 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.507670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5af5f1-5619-48ec-a323-ffc404a53057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.509570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5af5f1-5619-48ec-a323-ffc404a53057 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.538758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1046/2083] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:51 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:51.544443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5af5f1-5619-48ec-a323-ffc404a53057 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.544540 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:51.544636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6a5af5f1-5619-48ec-a323-ffc404a53057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.553759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7fd85826edf485b9c928bd810b14a97', 'request_id': 'req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.562012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7fd85826edf485b9c928bd810b14a97', 'request_id': 'req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.567853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7fd85826edf485b9c928bd810b14a97', 'request_id': 'req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.579982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] The token cache is being invalidate because project e7fd85826edf485b9c928bd810b14a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:51.580278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.580983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.582519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1038/2084] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:51 2022] DELETE /identity/v3/projects/e7fd85826edf485b9c928bd810b14a97 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:51.583256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.587968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.617107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5af5f1-5619-48ec-a323-ffc404a53057 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.622243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.622877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.623159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.623437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] PATH_INFO: `/v3/projects/d9288b120436428e86b5ff4e87f5d349` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.624565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9288b120436428e86b5ff4e87f5d349)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.626579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.659441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ed2c6-4b3d-4355-8b51-ce5b91d15cbe admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.664577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ffd78-93eb-4d69-be38-8de6917c54b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.665186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ffd78-93eb-4d69-be38-8de6917c54b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.665464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ffd78-93eb-4d69-be38-8de6917c54b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.665731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ffd78-93eb-4d69-be38-8de6917c54b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.666406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ffd78-93eb-4d69-be38-8de6917c54b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.668034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ffd78-93eb-4d69-be38-8de6917c54b4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.675719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9288b120436428e86b5ff4e87f5d349', 'request_id': 'req-5a305121-9550-4829-8e4b-ae9e931a036f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.678213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1039/2085] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:51.678987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ffd78-93eb-4d69-be38-8de6917c54b4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.680703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-987ffd78-93eb-4d69-be38-8de6917c54b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.681636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9288b120436428e86b5ff4e87f5d349', 'request_id': 'req-5a305121-9550-4829-8e4b-ae9e931a036f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.683417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:51.691693 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9288b120436428e86b5ff4e87f5d349', 'request_id': 'req-5a305121-9550-4829-8e4b-ae9e931a036f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.694406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ffd78-93eb-4d69-be38-8de6917c54b4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.705488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] The token cache is being invalidate because project d9288b120436428e86b5ff4e87f5d349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:51.705785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.706506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.709323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1047/2086] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:51 2022] DELETE /identity/v3/projects/d9288b120436428e86b5ff4e87f5d349 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:51.709471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.713660 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.761236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4674fe8-ff60-42db-89a7-88ca33bae6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.761866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4674fe8-ff60-42db-89a7-88ca33bae6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.762179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4674fe8-ff60-42db-89a7-88ca33bae6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.762453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4674fe8-ff60-42db-89a7-88ca33bae6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.763132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4674fe8-ff60-42db-89a7-88ca33bae6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.765083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4674fe8-ff60-42db-89a7-88ca33bae6c9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.789880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a305121-9550-4829-8e4b-ae9e931a036f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.793589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1040/2087] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:51 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:51.793589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4674fe8-ff60-42db-89a7-88ca33bae6c9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.793589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b4674fe8-ff60-42db-89a7-88ca33bae6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.799778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.800549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.800911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.801638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] PATH_INFO: `/v3/users/cab3d6ffd9464f708fa14c55d3fcc937` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.802530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cab3d6ffd9464f708fa14c55d3fcc937)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.804798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.805056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4674fe8-ff60-42db-89a7-88ca33bae6c9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.809993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29c401-ad6e-491e-9193-29d0fac59611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.813478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c401-ad6e-491e-9193-29d0fac59611 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.813478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c401-ad6e-491e-9193-29d0fac59611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.813478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c401-ad6e-491e-9193-29d0fac59611 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.813478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29c401-ad6e-491e-9193-29d0fac59611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.813478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29c401-ad6e-491e-9193-29d0fac59611 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.824952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1041/2088] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:51.826044 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c929f3-896f-4079-9e73-47e958c0f5ba None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.826337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c929f3-896f-4079-9e73-47e958c0f5ba None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.826617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c929f3-896f-4079-9e73-47e958c0f5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.847013 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f4c929f3-896f-4079-9e73-47e958c0f5ba None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:51.848413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f4c929f3-896f-4079-9e73-47e958c0f5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.863715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1042/2089] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:51.863715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c929f3-896f-4079-9e73-47e958c0f5ba None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.863715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f4c929f3-896f-4079-9e73-47e958c0f5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.871622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cab3d6ffd9464f708fa14c55d3fcc937'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.872093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cab3d6ffd9464f708fa14c55d3fcc937'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.872778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c929f3-896f-4079-9e73-47e958c0f5ba None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.877736 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab6a99-d6e2-4807-8815-9a266a32e07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:51.878412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab6a99-d6e2-4807-8815-9a266a32e07b admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.878696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab6a99-d6e2-4807-8815-9a266a32e07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.879020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab6a99-d6e2-4807-8815-9a266a32e07b admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.879623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab6a99-d6e2-4807-8815-9a266a32e07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:51.881225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab6a99-d6e2-4807-8815-9a266a32e07b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:51.882030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cab3d6ffd9464f708fa14c55d3fcc937'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.882876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cab3d6ffd9464f708fa14c55d3fcc937'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.893295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-07d065ec-9d17-4e3c-9ceb-3fe5a1cb9b43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cab3d6ffd9464f708fa14c55d3fcc937'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:51.902075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1043/2090] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:51.902075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab6a99-d6e2-4807-8815-9a266a32e07b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:51.902331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-06ab6a99-d6e2-4807-8815-9a266a32e07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.916919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab6a99-d6e2-4807-8815-9a266a32e07b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:51.918159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1048/2091] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:51 2022] DELETE /identity/v3/users/cab3d6ffd9464f708fa14c55d3fcc937 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:51.919442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570280e1-8848-4ebb-9290-1f4eea00caf4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:51.919722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570280e1-8848-4ebb-9290-1f4eea00caf4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:51.923507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570280e1-8848-4ebb-9290-1f4eea00caf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:51.944911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-570280e1-8848-4ebb-9290-1f4eea00caf4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:51.946778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-570280e1-8848-4ebb-9290-1f4eea00caf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:51.950612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:52.003241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a2827-6794-43be-abab-78a890195483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.003951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a2827-6794-43be-abab-78a890195483 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.004236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a2827-6794-43be-abab-78a890195483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.004513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a2827-6794-43be-abab-78a890195483 admin admin] PATH_INFO: `/v3/projects/084595e936a84b54a0d0e508fe0383d1/users/010ab9e117be48bf9f6083fc02d3ecba/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.029313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a2827-6794-43be-abab-78a890195483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=084595e936a84b54a0d0e508fe0383d1, user_id=010ab9e117be48bf9f6083fc02d3ecba, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.035506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1049/2092] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:52.036206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a2827-6794-43be-abab-78a890195483 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.036304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570280e1-8848-4ebb-9290-1f4eea00caf4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.036304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-570280e1-8848-4ebb-9290-1f4eea00caf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.053491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:52.059116 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570280e1-8848-4ebb-9290-1f4eea00caf4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.064123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1044/2093] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:51 2022] PUT /identity/v3/projects/084595e936a84b54a0d0e508fe0383d1/users/010ab9e117be48bf9f6083fc02d3ecba/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:52.067405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a2827-6794-43be-abab-78a890195483 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.140983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720f613-ddb9-4f39-a9a2-da39a53eecf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.142169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720f613-ddb9-4f39-a9a2-da39a53eecf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.143677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720f613-ddb9-4f39-a9a2-da39a53eecf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.143973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720f613-ddb9-4f39-a9a2-da39a53eecf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.144637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720f613-ddb9-4f39-a9a2-da39a53eecf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.146499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720f613-ddb9-4f39-a9a2-da39a53eecf1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.155391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a2827-6794-43be-abab-78a890195483 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.161361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1050/2094] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:52.161361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d01bf-9db9-4e5e-b113-d55427841ad2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.161361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d01bf-9db9-4e5e-b113-d55427841ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.161361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d01bf-9db9-4e5e-b113-d55427841ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.163042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4053f-bd5d-49c0-93ef-a25d048366a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.163841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4053f-bd5d-49c0-93ef-a25d048366a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.164454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4053f-bd5d-49c0-93ef-a25d048366a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.164596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4053f-bd5d-49c0-93ef-a25d048366a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.165303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4053f-bd5d-49c0-93ef-a25d048366a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:52.165686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4053f-bd5d-49c0-93ef-a25d048366a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.169345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4053f-bd5d-49c0-93ef-a25d048366a1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.182162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1045/2095] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:28:52.182983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4053f-bd5d-49c0-93ef-a25d048366a1 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.193228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4053f-bd5d-49c0-93ef-a25d048366a1 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.196724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-676d01bf-9db9-4e5e-b113-d55427841ad2 None None] MFA Rules not processed for user `010ab9e117be48bf9f6083fc02d3ecba`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:52.198042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-676d01bf-9db9-4e5e-b113-d55427841ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.198782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87412f7-8345-45ed-a518-9dfeeed597a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.199570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87412f7-8345-45ed-a518-9dfeeed597a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.200036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87412f7-8345-45ed-a518-9dfeeed597a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.200810 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87412f7-8345-45ed-a518-9dfeeed597a4 admin admin] PATH_INFO: `/v3/projects/3131c1978b9748e1bee2fa8a974bfdeb/users/29f82f899fa64b8eafaf966e6a83f600/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.219732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87412f7-8345-45ed-a518-9dfeeed597a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3131c1978b9748e1bee2fa8a974bfdeb, user_id=29f82f899fa64b8eafaf966e6a83f600, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.222793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87412f7-8345-45ed-a518-9dfeeed597a4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.238899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1046/2096] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:52 2022] PUT /identity/v3/projects/3131c1978b9748e1bee2fa8a974bfdeb/users/29f82f899fa64b8eafaf966e6a83f600/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:52.239783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87412f7-8345-45ed-a518-9dfeeed597a4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.284922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1051/2097] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:52.286153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725753-34c7-4246-9fbb-5a6f74ca0d63 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.286497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725753-34c7-4246-9fbb-5a6f74ca0d63 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.286841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725753-34c7-4246-9fbb-5a6f74ca0d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.318182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-71725753-34c7-4246-9fbb-5a6f74ca0d63 None None] MFA Rules not processed for user `29f82f899fa64b8eafaf966e6a83f600`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:52.318818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87412f7-8345-45ed-a518-9dfeeed597a4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.319399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-71725753-34c7-4246-9fbb-5a6f74ca0d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.324253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbee1e-1a29-47ec-842a-89f7d69f618c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.324865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbee1e-1a29-47ec-842a-89f7d69f618c admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.325146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbee1e-1a29-47ec-842a-89f7d69f618c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.325421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbee1e-1a29-47ec-842a-89f7d69f618c admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.326089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbee1e-1a29-47ec-842a-89f7d69f618c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.327986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbee1e-1a29-47ec-842a-89f7d69f618c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.356310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1047/2098] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:52 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:52.357095 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbee1e-1a29-47ec-842a-89f7d69f618c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.360366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fdcbee1e-1a29-47ec-842a-89f7d69f618c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.365679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:52.393821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1052/2099] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:52.394887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71725753-34c7-4246-9fbb-5a6f74ca0d63 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.409269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71725753-34c7-4246-9fbb-5a6f74ca0d63 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.427422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:52.435517 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbee1e-1a29-47ec-842a-89f7d69f618c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.440666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.441306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.441592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.441870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] PATH_INFO: `/v3/projects/83e92af2602e4c6dadd6adce44e09aba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.443154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83e92af2602e4c6dadd6adce44e09aba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.445073 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.469584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed666e0-98d8-466d-af37-1902076df831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.470238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed666e0-98d8-466d-af37-1902076df831 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.470527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed666e0-98d8-466d-af37-1902076df831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.470807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed666e0-98d8-466d-af37-1902076df831 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.471448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed666e0-98d8-466d-af37-1902076df831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.473392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed666e0-98d8-466d-af37-1902076df831 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.496108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83e92af2602e4c6dadd6adce44e09aba', 'request_id': 'req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.503592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83e92af2602e4c6dadd6adce44e09aba', 'request_id': 'req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.508151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1053/2100] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:52 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:52.508151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed666e0-98d8-466d-af37-1902076df831 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.512513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83e92af2602e4c6dadd6adce44e09aba', 'request_id': 'req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.524425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] The token cache is being invalidate because project 83e92af2602e4c6dadd6adce44e09aba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:52.524716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.525831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.527325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1048/2101] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:52 2022] DELETE /identity/v3/projects/83e92af2602e4c6dadd6adce44e09aba => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:52.527956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.531257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.572332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed666e0-98d8-466d-af37-1902076df831 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.577043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bef7bd-6eee-4661-8c68-8052a5e756a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.577651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef7bd-6eee-4661-8c68-8052a5e756a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.577953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef7bd-6eee-4661-8c68-8052a5e756a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.578233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef7bd-6eee-4661-8c68-8052a5e756a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.578926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bef7bd-6eee-4661-8c68-8052a5e756a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.580771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bef7bd-6eee-4661-8c68-8052a5e756a9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.609077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1054/2102] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:52 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:52.611004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bef7bd-6eee-4661-8c68-8052a5e756a9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.613635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f0bef7bd-6eee-4661-8c68-8052a5e756a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.621083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b4203-bd39-4b1e-82b1-dc2f3e431f15 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.626975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bef7bd-6eee-4661-8c68-8052a5e756a9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.631822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85fd5c-7cf9-470c-bc0c-0d6626269547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.632417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85fd5c-7cf9-470c-bc0c-0d6626269547 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.632807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85fd5c-7cf9-470c-bc0c-0d6626269547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.633084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85fd5c-7cf9-470c-bc0c-0d6626269547 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.633860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85fd5c-7cf9-470c-bc0c-0d6626269547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.635963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85fd5c-7cf9-470c-bc0c-0d6626269547 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.649809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:52.659343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1055/2103] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:52 2022] POST /identity/v3/users => generated 297 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:52.659715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85fd5c-7cf9-470c-bc0c-0d6626269547 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.661313 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1d85fd5c-7cf9-470c-bc0c-0d6626269547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.674668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85fd5c-7cf9-470c-bc0c-0d6626269547 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.677565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5305f8a5d2b540de91d3caedad454bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.678212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.678495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.678769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.679494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.680110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.680950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.681249 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] PATH_INFO: `/v3/users/4b98949b370a41f1abd9c60362098606` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.682158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.682415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b98949b370a41f1abd9c60362098606)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.684327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.734433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b98949b370a41f1abd9c60362098606'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.735119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b98949b370a41f1abd9c60362098606'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.743826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b98949b370a41f1abd9c60362098606'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.744493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b98949b370a41f1abd9c60362098606'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.748641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.750139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.752324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b98949b370a41f1abd9c60362098606'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:52.759863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=83,keystone:INSERT=8,keystone:DELETE=9 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:52.762714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1049/2104] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:52.763548 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.766886 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.777465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1056/2105] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:52 2022] DELETE /identity/v3/users/4b98949b370a41f1abd9c60362098606 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:52.777465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.777465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.785814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563071e-c716-4a90-b68d-1eaba15e99f6 service glance] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.864509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36883ce5-eba3-4711-bc94-09a9e256cc6a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.867254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6274f43-9066-42e5-8f1c-f5ef7cf46aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.868766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6274f43-9066-42e5-8f1c-f5ef7cf46aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.869054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6274f43-9066-42e5-8f1c-f5ef7cf46aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.869819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6274f43-9066-42e5-8f1c-f5ef7cf46aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.870799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6274f43-9066-42e5-8f1c-f5ef7cf46aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.872734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f264f-895a-494d-a53f-e60c4fec279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.873483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6274f43-9066-42e5-8f1c-f5ef7cf46aa4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.873603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f264f-895a-494d-a53f-e60c4fec279a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.874038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f264f-895a-494d-a53f-e60c4fec279a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.874516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f264f-895a-494d-a53f-e60c4fec279a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.875193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f264f-895a-494d-a53f-e60c4fec279a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.877231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f264f-895a-494d-a53f-e60c4fec279a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.892356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1050/2106] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:52.893614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1057/2107] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:52.894292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6274f43-9066-42e5-8f1c-f5ef7cf46aa4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.897356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f264f-895a-494d-a53f-e60c4fec279a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.902471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c6274f43-9066-42e5-8f1c-f5ef7cf46aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:52.909305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f264f-895a-494d-a53f-e60c4fec279a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.915300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd85f7f-1167-4fa0-8e19-299435aed49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.915894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd85f7f-1167-4fa0-8e19-299435aed49b admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.916217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd85f7f-1167-4fa0-8e19-299435aed49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.916486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd85f7f-1167-4fa0-8e19-299435aed49b admin admin] PATH_INFO: `/v3/projects/09297a4acaba4ceabe7f5b96aa209625/users/69fd7dec9cec446b89e68d9474833485/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.927055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6274f43-9066-42e5-8f1c-f5ef7cf46aa4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:52.935227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631facaf-1a5e-49fd-a37c-5f91231c0dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:52.935852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631facaf-1a5e-49fd-a37c-5f91231c0dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:52.936086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd85f7f-1167-4fa0-8e19-299435aed49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=09297a4acaba4ceabe7f5b96aa209625, user_id=69fd7dec9cec446b89e68d9474833485, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.936194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631facaf-1a5e-49fd-a37c-5f91231c0dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:52.936397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631facaf-1a5e-49fd-a37c-5f91231c0dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:52.937041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631facaf-1a5e-49fd-a37c-5f91231c0dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:52.938755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631facaf-1a5e-49fd-a37c-5f91231c0dd5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.940332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd85f7f-1167-4fa0-8e19-299435aed49b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:52.966900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1058/2108] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:52 2022] PUT /identity/v3/projects/09297a4acaba4ceabe7f5b96aa209625/users/69fd7dec9cec446b89e68d9474833485/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:52.968405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1051/2109] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:52.974922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631facaf-1a5e-49fd-a37c-5f91231c0dd5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.981521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd85f7f-1167-4fa0-8e19-299435aed49b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:52.998506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631facaf-1a5e-49fd-a37c-5f91231c0dd5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:53.089330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f4ac9-1d98-479e-a44e-3c97cf90e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:53.091479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f4ac9-1d98-479e-a44e-3c97cf90e9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.091762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f4ac9-1d98-479e-a44e-3c97cf90e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.092035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f4ac9-1d98-479e-a44e-3c97cf90e9c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.092679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f4ac9-1d98-479e-a44e-3c97cf90e9c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:53.094668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f4ac9-1d98-479e-a44e-3c97cf90e9c4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:53.117923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1052/2110] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:53.118862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd85f7f-1167-4fa0-8e19-299435aed49b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:53.122803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f4ac9-1d98-479e-a44e-3c97cf90e9c4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:53.133036 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:53.136965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c262b1-d499-422d-8091-a081c4d6aefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:53.137739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c262b1-d499-422d-8091-a081c4d6aefb admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.138128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c262b1-d499-422d-8091-a081c4d6aefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.138498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c262b1-d499-422d-8091-a081c4d6aefb admin admin] PATH_INFO: `/v3/projects/4c3f0c7489be4d2291ec80d969e3a584/users/61a257ce292c4c7fa40fe35dbce1964c/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.139331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f4ac9-1d98-479e-a44e-3c97cf90e9c4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:53.146194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba6f00-8f38-4ac2-92c6-6ea845a4a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:53.146962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba6f00-8f38-4ac2-92c6-6ea845a4a870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.147316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba6f00-8f38-4ac2-92c6-6ea845a4a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.147669 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba6f00-8f38-4ac2-92c6-6ea845a4a870 admin admin] PATH_INFO: `/v3/projects/09297a4acaba4ceabe7f5b96aa209625/users/69fd7dec9cec446b89e68d9474833485/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.150390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba6f00-8f38-4ac2-92c6-6ea845a4a870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09297a4acaba4ceabe7f5b96aa209625, user_id=69fd7dec9cec446b89e68d9474833485, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:53.154141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba6f00-8f38-4ac2-92c6-6ea845a4a870 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:53.164211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c262b1-d499-422d-8091-a081c4d6aefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3f0c7489be4d2291ec80d969e3a584, user_id=61a257ce292c4c7fa40fe35dbce1964c, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:53.170884 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c262b1-d499-422d-8091-a081c4d6aefb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:53.171466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1053/2111] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:53 2022] PUT /identity/v3/projects/09297a4acaba4ceabe7f5b96aa209625/users/69fd7dec9cec446b89e68d9474833485/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:53.182506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd515b1-7ea3-4fa5-92b3-bcde7a0e0d5f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.182792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd515b1-7ea3-4fa5-92b3-bcde7a0e0d5f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.183163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd515b1-7ea3-4fa5-92b3-bcde7a0e0d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.201206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1059/2112] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:52 2022] PUT /identity/v3/projects/4c3f0c7489be4d2291ec80d969e3a584/users/61a257ce292c4c7fa40fe35dbce1964c/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:53.207149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d830-a20c-4a9c-a1aa-c8a9a5222d1f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.207543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d830-a20c-4a9c-a1aa-c8a9a5222d1f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.208187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d830-a20c-4a9c-a1aa-c8a9a5222d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.238305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-49e2d830-a20c-4a9c-a1aa-c8a9a5222d1f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:53.239338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ffd515b1-7ea3-4fa5-92b3-bcde7a0e0d5f None None] MFA Rules not processed for user `69fd7dec9cec446b89e68d9474833485`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:53.239839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-49e2d830-a20c-4a9c-a1aa-c8a9a5222d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:53.241681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ffd515b1-7ea3-4fa5-92b3-bcde7a0e0d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:53.312443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:53.349511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1060/2113] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:53.352177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1054/2114] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:53.366332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd515b1-7ea3-4fa5-92b3-bcde7a0e0d5f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:53.367505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d830-a20c-4a9c-a1aa-c8a9a5222d1f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:53.368866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ffd515b1-7ea3-4fa5-92b3-bcde7a0e0d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:53.370945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-49e2d830-a20c-4a9c-a1aa-c8a9a5222d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:53.506015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d830-a20c-4a9c-a1aa-c8a9a5222d1f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:53.512876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88222184-930f-47d2-8f4f-e47dd949bdb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:53.513286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd515b1-7ea3-4fa5-92b3-bcde7a0e0d5f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:53.514182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88222184-930f-47d2-8f4f-e47dd949bdb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.514531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88222184-930f-47d2-8f4f-e47dd949bdb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.514870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88222184-930f-47d2-8f4f-e47dd949bdb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.517290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6aede-e774-48d0-b536-290f34029196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:53.517900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6aede-e774-48d0-b536-290f34029196 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.518200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6aede-e774-48d0-b536-290f34029196 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.518476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6aede-e774-48d0-b536-290f34029196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.530692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88222184-930f-47d2-8f4f-e47dd949bdb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:53.530949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6aede-e774-48d0-b536-290f34029196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:53.532380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88222184-930f-47d2-8f4f-e47dd949bdb7 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:53.532495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6aede-e774-48d0-b536-290f34029196 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:53.543211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1055/2115] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:28:53.544102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6aede-e774-48d0-b536-290f34029196 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:53.545363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6ef6aede-e774-48d0-b536-290f34029196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:53.545893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1061/2116] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:53.571667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6aede-e774-48d0-b536-290f34029196 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:53.576727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:53.577362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.577641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.577913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] PATH_INFO: `/v3/projects/538670da7cd4440ebf6446c375a9f103` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.579085 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=538670da7cd4440ebf6446c375a9f103)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:53.580991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:53.658676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538670da7cd4440ebf6446c375a9f103', 'request_id': 'req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:53.677671 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88222184-930f-47d2-8f4f-e47dd949bdb7 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:53.680881 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538670da7cd4440ebf6446c375a9f103', 'request_id': 'req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:53.689742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538670da7cd4440ebf6446c375a9f103', 'request_id': 'req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:53.693406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88222184-930f-47d2-8f4f-e47dd949bdb7 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:53.706076 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] The token cache is being invalidate because project 538670da7cd4440ebf6446c375a9f103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:53.706370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:53.707420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c7c9abdc-ca84-42c4-97cb-1988f2b1e8a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:53.710151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1056/2117] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:53 2022] DELETE /identity/v3/projects/538670da7cd4440ebf6446c375a9f103 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:53.715956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5e05e-051e-4b07-834f-e76195b4a21c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.716960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5e05e-051e-4b07-834f-e76195b4a21c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.717249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5e05e-051e-4b07-834f-e76195b4a21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.729362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:53.748933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-24d5e05e-051e-4b07-834f-e76195b4a21c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:53.751402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-24d5e05e-051e-4b07-834f-e76195b4a21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:53.777899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca0257-9f40-410a-8568-babf1ac82b09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:53.778824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca0257-9f40-410a-8568-babf1ac82b09 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.779415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca0257-9f40-410a-8568-babf1ac82b09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.779944 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca0257-9f40-410a-8568-babf1ac82b09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.783247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-77ca0257-9f40-410a-8568-babf1ac82b09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:53.835897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1057/2118] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:53.844388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7af93-e97d-4b18-9c74-b209fa07118b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.844875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7af93-e97d-4b18-9c74-b209fa07118b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.845446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7af93-e97d-4b18-9c74-b209fa07118b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.874094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8ba7af93-e97d-4b18-9c74-b209fa07118b None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:53.876069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8ba7af93-e97d-4b18-9c74-b209fa07118b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:53.884353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca0257-9f40-410a-8568-babf1ac82b09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:53.885498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1058/2119] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:28:53.886302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca0257-9f40-410a-8568-babf1ac82b09 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:53.889948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7af93-e97d-4b18-9c74-b209fa07118b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:53.899912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7af93-e97d-4b18-9c74-b209fa07118b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:53.900549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1062/2120] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:53.904910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54760e0f-cd7a-4c6b-a840-31e8262e4cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:53.905684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54760e0f-cd7a-4c6b-a840-31e8262e4cba admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.906470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54760e0f-cd7a-4c6b-a840-31e8262e4cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.906909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54760e0f-cd7a-4c6b-a840-31e8262e4cba admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.907737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54760e0f-cd7a-4c6b-a840-31e8262e4cba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:53.908145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54760e0f-cd7a-4c6b-a840-31e8262e4cba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:53.910235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54760e0f-cd7a-4c6b-a840-31e8262e4cba admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:53.924118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1059/2121] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:28:53.928852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca0257-9f40-410a-8568-babf1ac82b09 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:53.943781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca0257-9f40-410a-8568-babf1ac82b09 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:53.964731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067e45-331a-4c96-9ef3-611dc45926cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:53.965531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067e45-331a-4c96-9ef3-611dc45926cf admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:53.967574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067e45-331a-4c96-9ef3-611dc45926cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:53.968099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067e45-331a-4c96-9ef3-611dc45926cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:53.969451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067e45-331a-4c96-9ef3-611dc45926cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:53.973141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067e45-331a-4c96-9ef3-611dc45926cf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:54.014873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1063/2122] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:53 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:54.019662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54760e0f-cd7a-4c6b-a840-31e8262e4cba admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:54.036214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54760e0f-cd7a-4c6b-a840-31e8262e4cba admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:54.136524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76282972-20a1-4e9d-a186-851bd0725b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:54.137299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76282972-20a1-4e9d-a186-851bd0725b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:54.137648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76282972-20a1-4e9d-a186-851bd0725b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:54.138071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76282972-20a1-4e9d-a186-851bd0725b39 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:54.138863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76282972-20a1-4e9d-a186-851bd0725b39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:54.141433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76282972-20a1-4e9d-a186-851bd0725b39 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:54.175631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1060/2123] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:54 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:28:54.192617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067e45-331a-4c96-9ef3-611dc45926cf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:54.202929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067e45-331a-4c96-9ef3-611dc45926cf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:54.210248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268fa33-e71d-4fae-9052-4fb54e27609f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:54.210898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268fa33-e71d-4fae-9052-4fb54e27609f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:54.211172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268fa33-e71d-4fae-9052-4fb54e27609f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:54.211480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268fa33-e71d-4fae-9052-4fb54e27609f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:54.212131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268fa33-e71d-4fae-9052-4fb54e27609f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:54.213797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268fa33-e71d-4fae-9052-4fb54e27609f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:54.226800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1064/2124] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:28:54.231387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76282972-20a1-4e9d-a186-851bd0725b39 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:54.242271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76282972-20a1-4e9d-a186-851bd0725b39 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:54.255970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236211b-2292-4ea7-9ce6-adda6067be2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:54.256965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236211b-2292-4ea7-9ce6-adda6067be2e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:54.258637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236211b-2292-4ea7-9ce6-adda6067be2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:54.259117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236211b-2292-4ea7-9ce6-adda6067be2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:54.259802 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236211b-2292-4ea7-9ce6-adda6067be2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:54.261450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236211b-2292-4ea7-9ce6-adda6067be2e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:54.283252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1061/2125] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:54.286682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268fa33-e71d-4fae-9052-4fb54e27609f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:54.303748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268fa33-e71d-4fae-9052-4fb54e27609f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:54.314050 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36025b1f-357d-4b2c-9d1d-599136bc7e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:54.315162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36025b1f-357d-4b2c-9d1d-599136bc7e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:54.315501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36025b1f-357d-4b2c-9d1d-599136bc7e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:54.315855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36025b1f-357d-4b2c-9d1d-599136bc7e92 admin admin] PATH_INFO: `/v3/projects/eb6836d2a0994f27be0012f2608143f0/users/c6768e1bc2a7478c8c662a9c298d72e1/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:54.319798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:54.364173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36025b1f-357d-4b2c-9d1d-599136bc7e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6836d2a0994f27be0012f2608143f0, user_id=c6768e1bc2a7478c8c662a9c298d72e1, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:54.368898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36025b1f-357d-4b2c-9d1d-599136bc7e92 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:54.390460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1065/2126] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:54 2022] PUT /identity/v3/projects/eb6836d2a0994f27be0012f2608143f0/users/c6768e1bc2a7478c8c662a9c298d72e1/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:54.395395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b672a78-bdbb-47c4-8b6e-435d7f1e1923 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:54.395686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b672a78-bdbb-47c4-8b6e-435d7f1e1923 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:54.395966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b672a78-bdbb-47c4-8b6e-435d7f1e1923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:54.425659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5b672a78-bdbb-47c4-8b6e-435d7f1e1923 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:54.427646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5b672a78-bdbb-47c4-8b6e-435d7f1e1923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:54.520620 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1062/2127] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:54.528438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36025b1f-357d-4b2c-9d1d-599136bc7e92 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:54.530567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-36025b1f-357d-4b2c-9d1d-599136bc7e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:54.645322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36025b1f-357d-4b2c-9d1d-599136bc7e92 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:54.650175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db372b1a-c269-4430-afe9-d58d995885bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:54.650911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db372b1a-c269-4430-afe9-d58d995885bf service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:54.651336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db372b1a-c269-4430-afe9-d58d995885bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:54.651686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db372b1a-c269-4430-afe9-d58d995885bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:54.662422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db372b1a-c269-4430-afe9-d58d995885bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:54.664042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db372b1a-c269-4430-afe9-d58d995885bf service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:54.676993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1066/2128] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:56.098002 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de53f1-7db5-45ed-8ece-99f7b0c9f704 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:56.098655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de53f1-7db5-45ed-8ece-99f7b0c9f704 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:56.098655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de53f1-7db5-45ed-8ece-99f7b0c9f704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:56.106948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:56.133764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-29de53f1-7db5-45ed-8ece-99f7b0c9f704 None None] MFA Rules not processed for user `c6768e1bc2a7478c8c662a9c298d72e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:56.135070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-29de53f1-7db5-45ed-8ece-99f7b0c9f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:56.228841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1063/2129] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:56.244373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db372b1a-c269-4430-afe9-d58d995885bf service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:56.245871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-db372b1a-c269-4430-afe9-d58d995885bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:56.333585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db372b1a-c269-4430-afe9-d58d995885bf service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:56.338162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02a81a-de8f-4443-82b6-7b82e37fa9cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:56.353163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02a81a-de8f-4443-82b6-7b82e37fa9cd service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:56.353595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02a81a-de8f-4443-82b6-7b82e37fa9cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:56.353979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02a81a-de8f-4443-82b6-7b82e37fa9cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:56.364343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02a81a-de8f-4443-82b6-7b82e37fa9cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:56.365865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02a81a-de8f-4443-82b6-7b82e37fa9cd service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:56.391927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1067/2130] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:56.774550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de53f1-7db5-45ed-8ece-99f7b0c9f704 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:56.776151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-29de53f1-7db5-45ed-8ece-99f7b0c9f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:56.809625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de53f1-7db5-45ed-8ece-99f7b0c9f704 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:56.824013 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:56.824565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:56.824849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:56.825130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] PATH_INFO: `/v3/users/010ab9e117be48bf9f6083fc02d3ecba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:56.835711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] RBAC: Authorizing `identity:delete_user(user_id=010ab9e117be48bf9f6083fc02d3ecba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:56.837949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:56.978101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010ab9e117be48bf9f6083fc02d3ecba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:56.993422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010ab9e117be48bf9f6083fc02d3ecba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:56.994451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010ab9e117be48bf9f6083fc02d3ecba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:56.995245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010ab9e117be48bf9f6083fc02d3ecba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:57.005090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010ab9e117be48bf9f6083fc02d3ecba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:57.026951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1064/2131] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:56 2022] DELETE /identity/v3/users/010ab9e117be48bf9f6083fc02d3ecba => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:57.032120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b9b0-4f60-4d94-a114-71c5409ad9fe None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:57.032120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b9b0-4f60-4d94-a114-71c5409ad9fe None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:57.032356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b9b0-4f60-4d94-a114-71c5409ad9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:57.054374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d798b9b0-4f60-4d94-a114-71c5409ad9fe None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:57.056160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d798b9b0-4f60-4d94-a114-71c5409ad9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:57.120541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1068/2132] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:57.132612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:57.134985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:57.209169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d207c1-8ff5-409e-99e4-523143b3b71c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:57.217108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed6597-6b2b-4901-811f-8fc2f2fb6d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:57.217855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed6597-6b2b-4901-811f-8fc2f2fb6d7f service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:57.218237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed6597-6b2b-4901-811f-8fc2f2fb6d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:57.218914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed6597-6b2b-4901-811f-8fc2f2fb6d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:57.229183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed6597-6b2b-4901-811f-8fc2f2fb6d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:57.231827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed6597-6b2b-4901-811f-8fc2f2fb6d7f service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:57.258402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1065/2133] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:28:57.754605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b9b0-4f60-4d94-a114-71c5409ad9fe None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:57.756762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d798b9b0-4f60-4d94-a114-71c5409ad9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:57.773507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b9b0-4f60-4d94-a114-71c5409ad9fe None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:57.780870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:57.781922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:57.782566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:57.783347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] PATH_INFO: `/v3/projects/084595e936a84b54a0d0e508fe0383d1` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:57.785391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=084595e936a84b54a0d0e508fe0383d1)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:57.787729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:57.832492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:57.857448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '084595e936a84b54a0d0e508fe0383d1', 'request_id': 'req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:57.863972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '084595e936a84b54a0d0e508fe0383d1', 'request_id': 'req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:57.877079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '084595e936a84b54a0d0e508fe0383d1', 'request_id': 'req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:57.891376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] The token cache is being invalidate because project 084595e936a84b54a0d0e508fe0383d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:57.891768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:57.892507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-205868cb-2285-48e5-8a4e-a4a7d64bc5e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:57.897160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1069/2134] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:57 2022] DELETE /identity/v3/projects/084595e936a84b54a0d0e508fe0383d1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:57.898341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed6597-6b2b-4901-811f-8fc2f2fb6d7f service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:57.911359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed6597-6b2b-4901-811f-8fc2f2fb6d7f service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:57.915943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:57.990047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ce123-35cd-4705-9cf6-cb4dc4b2cf31 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:57.990333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ce123-35cd-4705-9cf6-cb4dc4b2cf31 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:57.990615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ce123-35cd-4705-9cf6-cb4dc4b2cf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:57.998162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:57.999506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:58.000343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:58.001058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] PATH_INFO: `/v3/users/69fd7dec9cec446b89e68d9474833485` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:58.002648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69fd7dec9cec446b89e68d9474833485)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:58.004816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:58.015246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2f1ce123-35cd-4705-9cf6-cb4dc4b2cf31 None None] MFA Rules not processed for user `c6768e1bc2a7478c8c662a9c298d72e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:58.016817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2f1ce123-35cd-4705-9cf6-cb4dc4b2cf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:58.102624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69fd7dec9cec446b89e68d9474833485'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.114543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69fd7dec9cec446b89e68d9474833485'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.115152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69fd7dec9cec446b89e68d9474833485'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.115987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69fd7dec9cec446b89e68d9474833485'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.140063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8f39c635-4312-4113-bd91-8e72f4c5ee31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69fd7dec9cec446b89e68d9474833485'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.167987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1066/2135] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:28:57 2022] DELETE /identity/v3/users/69fd7dec9cec446b89e68d9474833485 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:28:58.168488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1070/2136] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:58.581464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ce123-35cd-4705-9cf6-cb4dc4b2cf31 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:58.601729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2f1ce123-35cd-4705-9cf6-cb4dc4b2cf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:58.617661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ce123-35cd-4705-9cf6-cb4dc4b2cf31 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:58.705781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:58.706524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:58.706862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:58.707247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] PATH_INFO: `/v3/projects/09297a4acaba4ceabe7f5b96aa209625` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:58.709502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=09297a4acaba4ceabe7f5b96aa209625)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:58.712102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:58.776215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09297a4acaba4ceabe7f5b96aa209625', 'request_id': 'req-524e44a8-36da-47f0-8105-2e3ec9637f5b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.784269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09297a4acaba4ceabe7f5b96aa209625', 'request_id': 'req-524e44a8-36da-47f0-8105-2e3ec9637f5b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.789509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:58.798831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09297a4acaba4ceabe7f5b96aa209625', 'request_id': 'req-524e44a8-36da-47f0-8105-2e3ec9637f5b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.815187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] The token cache is being invalidate because project 09297a4acaba4ceabe7f5b96aa209625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:28:58.815482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.816129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-524e44a8-36da-47f0-8105-2e3ec9637f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:28:58.817987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1071/2137] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:28:58 2022] DELETE /identity/v3/projects/09297a4acaba4ceabe7f5b96aa209625 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:58.823339 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd9819-e236-4df0-a509-e7ecace4638e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:58.824675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd9819-e236-4df0-a509-e7ecace4638e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:58.825454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd9819-e236-4df0-a509-e7ecace4638e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:58.874113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6efd9819-e236-4df0-a509-e7ecace4638e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:58.875908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6efd9819-e236-4df0-a509-e7ecace4638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:58.957722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1067/2138] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:58.965422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a9b6c-5ba6-48cf-8526-cfc1016c0a8c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:58.966223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a9b6c-5ba6-48cf-8526-cfc1016c0a8c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:58.966223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a9b6c-5ba6-48cf-8526-cfc1016c0a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:58.988912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e27a9b6c-5ba6-48cf-8526-cfc1016c0a8c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:58.990186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e27a9b6c-5ba6-48cf-8526-cfc1016c0a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:59.002920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1072/2139] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:59.003471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd9819-e236-4df0-a509-e7ecace4638e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:59.014574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd9819-e236-4df0-a509-e7ecace4638e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:59.026232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42907ac9-8dcf-475d-a068-84714bd6bf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:59.026964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42907ac9-8dcf-475d-a068-84714bd6bf50 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:59.027359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42907ac9-8dcf-475d-a068-84714bd6bf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:59.027744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42907ac9-8dcf-475d-a068-84714bd6bf50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:59.028496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42907ac9-8dcf-475d-a068-84714bd6bf50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:28:59.029051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42907ac9-8dcf-475d-a068-84714bd6bf50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:59.030826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42907ac9-8dcf-475d-a068-84714bd6bf50 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:59.047333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1068/2140] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:28:59.051959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a9b6c-5ba6-48cf-8526-cfc1016c0a8c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:59.064792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a9b6c-5ba6-48cf-8526-cfc1016c0a8c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:59.070434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b106637-3c68-4098-acda-d864396eec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:59.071821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b106637-3c68-4098-acda-d864396eec7b admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:59.072110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b106637-3c68-4098-acda-d864396eec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:59.072655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b106637-3c68-4098-acda-d864396eec7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:59.073029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b106637-3c68-4098-acda-d864396eec7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:59.075224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b106637-3c68-4098-acda-d864396eec7b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:59.154543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1073/2141] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:59 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:59.156644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42907ac9-8dcf-475d-a068-84714bd6bf50 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:59.181075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42907ac9-8dcf-475d-a068-84714bd6bf50 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:59.222422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:28:59.325283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b05ebf-14c0-4587-881f-f64e7029052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:59.325990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b05ebf-14c0-4587-881f-f64e7029052f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:59.326326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b05ebf-14c0-4587-881f-f64e7029052f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:59.326598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b05ebf-14c0-4587-881f-f64e7029052f admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:59.327409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b05ebf-14c0-4587-881f-f64e7029052f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:59.329513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b05ebf-14c0-4587-881f-f64e7029052f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:59.393244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1069/2142] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:28:59 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:28:59.397636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b106637-3c68-4098-acda-d864396eec7b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:59.421168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b106637-3c68-4098-acda-d864396eec7b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:59.441156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49721df5-ded2-4e34-82e9-fcfd78994d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:59.441811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49721df5-ded2-4e34-82e9-fcfd78994d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:59.442090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49721df5-ded2-4e34-82e9-fcfd78994d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:59.442362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49721df5-ded2-4e34-82e9-fcfd78994d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:59.446274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49721df5-ded2-4e34-82e9-fcfd78994d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:59.446274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49721df5-ded2-4e34-82e9-fcfd78994d24 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:59.459298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1074/2143] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:59.460992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b05ebf-14c0-4587-881f-f64e7029052f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:59.472587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b05ebf-14c0-4587-881f-f64e7029052f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:59.481424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa7cc6-629b-4499-8e22-fdff55ff4c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:59.482383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa7cc6-629b-4499-8e22-fdff55ff4c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:59.482997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa7cc6-629b-4499-8e22-fdff55ff4c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:59.483436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa7cc6-629b-4499-8e22-fdff55ff4c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:59.484661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa7cc6-629b-4499-8e22-fdff55ff4c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:59.487148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa7cc6-629b-4499-8e22-fdff55ff4c8b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:59.501749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1070/2144] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:28:59.505750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49721df5-ded2-4e34-82e9-fcfd78994d24 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:59.520663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49721df5-ded2-4e34-82e9-fcfd78994d24 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:59.534032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe85d9-7e76-4acb-9889-a961cf6b187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:59.535183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe85d9-7e76-4acb-9889-a961cf6b187c admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:59.535561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe85d9-7e76-4acb-9889-a961cf6b187c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:59.536018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe85d9-7e76-4acb-9889-a961cf6b187c admin admin] PATH_INFO: `/v3/projects/3ef5018179bd4e8da444721484a38051/users/483b2d7fe1a840f0b65a96eda151d318/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:59.559960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe85d9-7e76-4acb-9889-a961cf6b187c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef5018179bd4e8da444721484a38051, user_id=483b2d7fe1a840f0b65a96eda151d318, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:59.563669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe85d9-7e76-4acb-9889-a961cf6b187c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:59.597952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1075/2145] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:28:59 2022] PUT /identity/v3/projects/3ef5018179bd4e8da444721484a38051/users/483b2d7fe1a840f0b65a96eda151d318/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:28:59.609092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0008c7d-4e4b-4268-a11c-58e222018cba None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:59.609317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0008c7d-4e4b-4268-a11c-58e222018cba None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:59.609794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0008c7d-4e4b-4268-a11c-58e222018cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:59.676237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a0008c7d-4e4b-4268-a11c-58e222018cba None None] MFA Rules not processed for user `483b2d7fe1a840f0b65a96eda151d318`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:59.677821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a0008c7d-4e4b-4268-a11c-58e222018cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:59.745138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b0c10-1d08-47ca-9150-76c5c8a6ef10 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:59.745403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b0c10-1d08-47ca-9150-76c5c8a6ef10 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:59.747302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b0c10-1d08-47ca-9150-76c5c8a6ef10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:59.791504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-448b0c10-1d08-47ca-9150-76c5c8a6ef10 None None] MFA Rules not processed for user `61a257ce292c4c7fa40fe35dbce1964c`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:28:59.793619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-448b0c10-1d08-47ca-9150-76c5c8a6ef10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:28:59.830481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1071/2146] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:59.835489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0008c7d-4e4b-4268-a11c-58e222018cba None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:59.923399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0008c7d-4e4b-4268-a11c-58e222018cba None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:28:59.934005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1076/2147] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:28:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:28:59.935577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e06c-993f-4bc0-a6eb-87914ef6247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:28:59.936404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e06c-993f-4bc0-a6eb-87914ef6247d admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:28:59.936730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e06c-993f-4bc0-a6eb-87914ef6247d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:28:59.937078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e06c-993f-4bc0-a6eb-87914ef6247d admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:28:59.937851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32e06c-993f-4bc0-a6eb-87914ef6247d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:28:59.939946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32e06c-993f-4bc0-a6eb-87914ef6247d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:28:59.948390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b0c10-1d08-47ca-9150-76c5c8a6ef10 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:28:59.954406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-448b0c10-1d08-47ca-9150-76c5c8a6ef10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:00.011068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1072/2148] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:28:59 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:29:00.018492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e06c-993f-4bc0-a6eb-87914ef6247d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:00.045865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:00.119559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e06c-993f-4bc0-a6eb-87914ef6247d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:00.121356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b0c10-1d08-47ca-9150-76c5c8a6ef10 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:00.126147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171814c7-fafd-4f48-8700-cf2f979a7c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:00.127601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171814c7-fafd-4f48-8700-cf2f979a7c83 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:00.128950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171814c7-fafd-4f48-8700-cf2f979a7c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:00.129469 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171814c7-fafd-4f48-8700-cf2f979a7c83 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:00.130459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171814c7-fafd-4f48-8700-cf2f979a7c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:00.132634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:00.134138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171814c7-fafd-4f48-8700-cf2f979a7c83 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:00.166126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1073/2149] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:29:00 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:29:00.170777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171814c7-fafd-4f48-8700-cf2f979a7c83 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:00.185884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171814c7-fafd-4f48-8700-cf2f979a7c83 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:00.192075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f14a16-24a6-4cb1-add8-e3bb19a456e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:00.192577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f14a16-24a6-4cb1-add8-e3bb19a456e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:00.192937 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f14a16-24a6-4cb1-add8-e3bb19a456e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:00.193281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f14a16-24a6-4cb1-add8-e3bb19a456e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:00.194052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f14a16-24a6-4cb1-add8-e3bb19a456e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:00.196335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f14a16-24a6-4cb1-add8-e3bb19a456e4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:00.213378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1074/2150] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:29:00.215716 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e02e05-1d37-4f50-8065-8edb95a5293f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:00.216848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e02e05-1d37-4f50-8065-8edb95a5293f service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:00.217243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e02e05-1d37-4f50-8065-8edb95a5293f service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:00.217558 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e02e05-1d37-4f50-8065-8edb95a5293f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:00.220208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f14a16-24a6-4cb1-add8-e3bb19a456e4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:00.235375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f14a16-24a6-4cb1-add8-e3bb19a456e4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:00.240516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272696f-5feb-4c65-9017-f7929b0072ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:00.241257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272696f-5feb-4c65-9017-f7929b0072ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:00.241605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272696f-5feb-4c65-9017-f7929b0072ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:00.241975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272696f-5feb-4c65-9017-f7929b0072ec admin admin] PATH_INFO: `/v3/projects/65f27f8cd5974ac99c01c17c5d70da6d/users/de656f06010f4814bf29e4505c073fac/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:00.267480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272696f-5feb-4c65-9017-f7929b0072ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f27f8cd5974ac99c01c17c5d70da6d, user_id=de656f06010f4814bf29e4505c073fac, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:00.270306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272696f-5feb-4c65-9017-f7929b0072ec admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:00.287546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1075/2151] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:29:00 2022] PUT /identity/v3/projects/65f27f8cd5974ac99c01c17c5d70da6d/users/de656f06010f4814bf29e4505c073fac/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:00.292869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272696f-5feb-4c65-9017-f7929b0072ec admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:00.295782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e02e05-1d37-4f50-8065-8edb95a5293f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:00.299465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e02e05-1d37-4f50-8065-8edb95a5293f service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:00.322030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1077/2152] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:28:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:29:00.398342 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272696f-5feb-4c65-9017-f7929b0072ec admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:00.408073 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dd719-781d-48cf-a9b7-37f279e45c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:00.408702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dd719-781d-48cf-a9b7-37f279e45c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:00.408982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dd719-781d-48cf-a9b7-37f279e45c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:00.409364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dd719-781d-48cf-a9b7-37f279e45c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:00.410256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dd719-781d-48cf-a9b7-37f279e45c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:00.412158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dd719-781d-48cf-a9b7-37f279e45c40 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:00.433674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1076/2153] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:29:00.439164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e02e05-1d37-4f50-8065-8edb95a5293f service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:00.466368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e02e05-1d37-4f50-8065-8edb95a5293f service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:00.485398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ccfdd-116a-4d0c-ae3e-556a421c4b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:00.486309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ccfdd-116a-4d0c-ae3e-556a421c4b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:00.486849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ccfdd-116a-4d0c-ae3e-556a421c4b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:00.487128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ccfdd-116a-4d0c-ae3e-556a421c4b12 admin admin] PATH_INFO: `/v3/projects/65f27f8cd5974ac99c01c17c5d70da6d/users/de656f06010f4814bf29e4505c073fac/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:00.491662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ccfdd-116a-4d0c-ae3e-556a421c4b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f27f8cd5974ac99c01c17c5d70da6d, user_id=de656f06010f4814bf29e4505c073fac, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:00.496962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ccfdd-116a-4d0c-ae3e-556a421c4b12 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:00.518383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1078/2154] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:29:00 2022] PUT /identity/v3/projects/65f27f8cd5974ac99c01c17c5d70da6d/users/de656f06010f4814bf29e4505c073fac/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:29:00.530833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245e761-aa62-4d48-ae89-a030e05159cb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:00.531124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245e761-aa62-4d48-ae89-a030e05159cb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:00.531523 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245e761-aa62-4d48-ae89-a030e05159cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:00.569459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3245e761-aa62-4d48-ae89-a030e05159cb None None] MFA Rules not processed for user `de656f06010f4814bf29e4505c073fac`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:00.571529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3245e761-aa62-4d48-ae89-a030e05159cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:00.644601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:00.691330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1077/2155] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:00.704053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ccfdd-116a-4d0c-ae3e-556a421c4b12 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:00.706073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d81ccfdd-116a-4d0c-ae3e-556a421c4b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:00.797792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ccfdd-116a-4d0c-ae3e-556a421c4b12 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:00.802103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8f9e7-3a31-415b-b67b-2b6abf0c8ffb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:00.802805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8f9e7-3a31-415b-b67b-2b6abf0c8ffb service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:00.803149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8f9e7-3a31-415b-b67b-2b6abf0c8ffb service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:00.803580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8f9e7-3a31-415b-b67b-2b6abf0c8ffb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:00.830526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8f9e7-3a31-415b-b67b-2b6abf0c8ffb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:00.830526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8f9e7-3a31-415b-b67b-2b6abf0c8ffb service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:00.854229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1079/2156] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:29:00.994760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245e761-aa62-4d48-ae89-a030e05159cb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:01.005348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245e761-aa62-4d48-ae89-a030e05159cb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:01.012089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8ce5f-a1bf-425c-b235-650aa572ba7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:01.012954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8ce5f-a1bf-425c-b235-650aa572ba7a service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:01.013306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8ce5f-a1bf-425c-b235-650aa572ba7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:01.013660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8ce5f-a1bf-425c-b235-650aa572ba7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:01.118869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8ce5f-a1bf-425c-b235-650aa572ba7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:01.120558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8ce5f-a1bf-425c-b235-650aa572ba7a service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:01.131696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1078/2157] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:29:01.439338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8f9e7-3a31-415b-b67b-2b6abf0c8ffb service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:01.490282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:01.517101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8f9e7-3a31-415b-b67b-2b6abf0c8ffb service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:01.522813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:01.524299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:01.525206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:01.525206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] PATH_INFO: `/v3/users/483b2d7fe1a840f0b65a96eda151d318` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:01.526625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=483b2d7fe1a840f0b65a96eda151d318)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:01.528775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:01.601467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '483b2d7fe1a840f0b65a96eda151d318'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:01.602704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '483b2d7fe1a840f0b65a96eda151d318'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:01.612464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '483b2d7fe1a840f0b65a96eda151d318'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:01.613306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '483b2d7fe1a840f0b65a96eda151d318'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:01.628440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '483b2d7fe1a840f0b65a96eda151d318'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:01.655997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1080/2158] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:29:01 2022] DELETE /identity/v3/users/483b2d7fe1a840f0b65a96eda151d318 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:01.660446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede9ea-cfd2-4d8f-8fd1-794f0f05d3db None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:01.662003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede9ea-cfd2-4d8f-8fd1-794f0f05d3db None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:01.662436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede9ea-cfd2-4d8f-8fd1-794f0f05d3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:01.682792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-94ede9ea-cfd2-4d8f-8fd1-794f0f05d3db None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:01.684230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-94ede9ea-cfd2-4d8f-8fd1-794f0f05d3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:01.778782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1079/2159] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:01.784600 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:01.786183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:01.856656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcd346-367f-4269-bfa9-a1b5acc8e856 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:01.861175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c199e-5087-4ed5-8e3f-d083ffc84962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:01.862197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c199e-5087-4ed5-8e3f-d083ffc84962 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:01.862461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c199e-5087-4ed5-8e3f-d083ffc84962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:01.862814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c199e-5087-4ed5-8e3f-d083ffc84962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:01.876695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c199e-5087-4ed5-8e3f-d083ffc84962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:01.877698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c199e-5087-4ed5-8e3f-d083ffc84962 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:01.888725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1081/2160] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:29:02.256165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ede9ea-cfd2-4d8f-8fd1-794f0f05d3db None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:02.257732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-94ede9ea-cfd2-4d8f-8fd1-794f0f05d3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:02.276188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ede9ea-cfd2-4d8f-8fd1-794f0f05d3db None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:02.283657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:02.284467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:02.284851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:02.285234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] PATH_INFO: `/v3/projects/3ef5018179bd4e8da444721484a38051` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:02.286630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ef5018179bd4e8da444721484a38051)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:02.289786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:02.358272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ef5018179bd4e8da444721484a38051', 'request_id': 'req-84c621bc-c0d2-4565-9671-51b5cde291f0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.377838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ef5018179bd4e8da444721484a38051', 'request_id': 'req-84c621bc-c0d2-4565-9671-51b5cde291f0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.398627 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ef5018179bd4e8da444721484a38051', 'request_id': 'req-84c621bc-c0d2-4565-9671-51b5cde291f0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.413852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] The token cache is being invalidate because project 3ef5018179bd4e8da444721484a38051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:29:02.414877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.416155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-84c621bc-c0d2-4565-9671-51b5cde291f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.418420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1080/2161] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:29:02 2022] DELETE /identity/v3/projects/3ef5018179bd4e8da444721484a38051 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:02.422282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c199e-5087-4ed5-8e3f-d083ffc84962 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:02.434760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c199e-5087-4ed5-8e3f-d083ffc84962 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:02.518894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:02.519535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:02.519817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:02.520093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] PATH_INFO: `/v3/users/de656f06010f4814bf29e4505c073fac` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:02.522261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] RBAC: Authorizing `identity:delete_user(user_id=de656f06010f4814bf29e4505c073fac)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:02.525158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:02.529422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:02.626505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de656f06010f4814bf29e4505c073fac'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.627321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de656f06010f4814bf29e4505c073fac'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.640447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de656f06010f4814bf29e4505c073fac'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.641358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de656f06010f4814bf29e4505c073fac'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.656351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de656f06010f4814bf29e4505c073fac'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:02.682456 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1082/2162] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:29:02 2022] DELETE /identity/v3/users/de656f06010f4814bf29e4505c073fac => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:02.988335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635b8a-015b-49a2-8ae6-cc587923050b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:02.988818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635b8a-015b-49a2-8ae6-cc587923050b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:02.988910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635b8a-015b-49a2-8ae6-cc587923050b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:02.997491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:03.015143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b8635b8a-015b-49a2-8ae6-cc587923050b None None] MFA Rules not processed for user `61a257ce292c4c7fa40fe35dbce1964c`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:03.016406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b8635b8a-015b-49a2-8ae6-cc587923050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:03.093392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:03.095119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:03.117183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1081/2163] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:03.214244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403ba42-1284-4d59-893d-41e42d08830c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:03.221090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:03.222184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:03.222682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:03.223156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] PATH_INFO: `/v3/projects/65f27f8cd5974ac99c01c17c5d70da6d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:03.225046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f27f8cd5974ac99c01c17c5d70da6d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:03.227301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:03.300301 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f27f8cd5974ac99c01c17c5d70da6d', 'request_id': 'req-927abec4-62f8-4826-8c31-36c05da45e5a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:03.305859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f27f8cd5974ac99c01c17c5d70da6d', 'request_id': 'req-927abec4-62f8-4826-8c31-36c05da45e5a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:03.326292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f27f8cd5974ac99c01c17c5d70da6d', 'request_id': 'req-927abec4-62f8-4826-8c31-36c05da45e5a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:03.348590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] The token cache is being invalidate because project 65f27f8cd5974ac99c01c17c5d70da6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:29:03.348841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:03.349650 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-927abec4-62f8-4826-8c31-36c05da45e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:03.352539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1083/2164] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:29:03 2022] DELETE /identity/v3/projects/65f27f8cd5974ac99c01c17c5d70da6d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:03.358466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae538-4867-4fda-99ed-f4b10dc4dfad None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:03.358758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae538-4867-4fda-99ed-f4b10dc4dfad None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:03.359031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae538-4867-4fda-99ed-f4b10dc4dfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:03.384703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-da3ae538-4867-4fda-99ed-f4b10dc4dfad None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:03.386586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-da3ae538-4867-4fda-99ed-f4b10dc4dfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:03.490567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1082/2165] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:03.503985 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a498e45-98de-4a19-a104-ea2d226a4027 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:03.504699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a498e45-98de-4a19-a104-ea2d226a4027 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:03.506150 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a498e45-98de-4a19-a104-ea2d226a4027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:03.539866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2a498e45-98de-4a19-a104-ea2d226a4027 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:03.541137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2a498e45-98de-4a19-a104-ea2d226a4027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:03.553406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1084/2166] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:03.558055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ae538-4867-4fda-99ed-f4b10dc4dfad None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:03.569296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ae538-4867-4fda-99ed-f4b10dc4dfad None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:03.576582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50bc59-2abc-4a14-9ed9-dcc6598744dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:03.577424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50bc59-2abc-4a14-9ed9-dcc6598744dc admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:03.577712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50bc59-2abc-4a14-9ed9-dcc6598744dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:03.578115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50bc59-2abc-4a14-9ed9-dcc6598744dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:03.578896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50bc59-2abc-4a14-9ed9-dcc6598744dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:29:03.579302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50bc59-2abc-4a14-9ed9-dcc6598744dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:03.583252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50bc59-2abc-4a14-9ed9-dcc6598744dc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:03.628364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1083/2167] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:29:03.633788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a498e45-98de-4a19-a104-ea2d226a4027 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:03.640425 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:03.652412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a498e45-98de-4a19-a104-ea2d226a4027 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:03.659103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e81220-ca9a-4c51-82a3-6e4f8913876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:03.660149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e81220-ca9a-4c51-82a3-6e4f8913876b admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:03.660829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e81220-ca9a-4c51-82a3-6e4f8913876b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:03.661282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e81220-ca9a-4c51-82a3-6e4f8913876b admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:03.662320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e81220-ca9a-4c51-82a3-6e4f8913876b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:03.667323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e81220-ca9a-4c51-82a3-6e4f8913876b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:03.711125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1085/2168] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:29:03 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:29:03.720964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50bc59-2abc-4a14-9ed9-dcc6598744dc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:03.733315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50bc59-2abc-4a14-9ed9-dcc6598744dc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:03.815580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309c49c-b18e-4802-a7fb-1a26512da7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:03.816284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309c49c-b18e-4802-a7fb-1a26512da7cc admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:03.816634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309c49c-b18e-4802-a7fb-1a26512da7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:03.817109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309c49c-b18e-4802-a7fb-1a26512da7cc admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:03.817826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309c49c-b18e-4802-a7fb-1a26512da7cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:03.820122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309c49c-b18e-4802-a7fb-1a26512da7cc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:03.849465 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1084/2169] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:29:03 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:29:03.854189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e81220-ca9a-4c51-82a3-6e4f8913876b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:03.871119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e81220-ca9a-4c51-82a3-6e4f8913876b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:03.878218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffe36f-d005-4ee4-9867-198ce66c9721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:03.878866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffe36f-d005-4ee4-9867-198ce66c9721 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:03.879145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffe36f-d005-4ee4-9867-198ce66c9721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:03.879416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffe36f-d005-4ee4-9867-198ce66c9721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:03.880068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffe36f-d005-4ee4-9867-198ce66c9721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:03.882018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffe36f-d005-4ee4-9867-198ce66c9721 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:03.897537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1086/2170] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:29:03.901393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309c49c-b18e-4802-a7fb-1a26512da7cc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:03.912441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309c49c-b18e-4802-a7fb-1a26512da7cc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:03.920135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47816b-2805-49b9-b61a-853b1db31aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:03.921115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47816b-2805-49b9-b61a-853b1db31aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:03.922272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47816b-2805-49b9-b61a-853b1db31aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:03.936629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47816b-2805-49b9-b61a-853b1db31aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:03.937547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47816b-2805-49b9-b61a-853b1db31aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:03.939733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47816b-2805-49b9-b61a-853b1db31aa9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:03.978010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1085/2171] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:29:03.980035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffe36f-d005-4ee4-9867-198ce66c9721 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:03.998085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffe36f-d005-4ee4-9867-198ce66c9721 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:04.004651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7ff32-c13c-4a97-8326-63293b581574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:04.007558 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7ff32-c13c-4a97-8326-63293b581574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:04.007936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7ff32-c13c-4a97-8326-63293b581574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:04.008285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7ff32-c13c-4a97-8326-63293b581574 admin admin] PATH_INFO: `/v3/projects/8185f0ff512e4dc192291cc4079ebb2b/users/6073b9546a834ac4bb8801e7ed6eb4db/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:04.028679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7ff32-c13c-4a97-8326-63293b581574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8185f0ff512e4dc192291cc4079ebb2b, user_id=6073b9546a834ac4bb8801e7ed6eb4db, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:04.033497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7ff32-c13c-4a97-8326-63293b581574 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:04.058883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1087/2172] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:29:03 2022] PUT /identity/v3/projects/8185f0ff512e4dc192291cc4079ebb2b/users/6073b9546a834ac4bb8801e7ed6eb4db/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:04.060930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c40d6b-bf09-4f87-8309-f1ea1dd72e62 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:04.061417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c40d6b-bf09-4f87-8309-f1ea1dd72e62 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:04.061717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c40d6b-bf09-4f87-8309-f1ea1dd72e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:04.094206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-06c40d6b-bf09-4f87-8309-f1ea1dd72e62 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:04.096687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-06c40d6b-bf09-4f87-8309-f1ea1dd72e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:04.116306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:04.186992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1086/2173] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:04.206651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7ff32-c13c-4a97-8326-63293b581574 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:04.212023 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-98e7ff32-c13c-4a97-8326-63293b581574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:04.307565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7ff32-c13c-4a97-8326-63293b581574 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:04.313274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52ef8-a3a4-4dc2-93e9-9d2ebb30f415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:04.314567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52ef8-a3a4-4dc2-93e9-9d2ebb30f415 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:04.315732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52ef8-a3a4-4dc2-93e9-9d2ebb30f415 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:04.316206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52ef8-a3a4-4dc2-93e9-9d2ebb30f415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:04.331267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a52ef8-a3a4-4dc2-93e9-9d2ebb30f415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:04.333803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a52ef8-a3a4-4dc2-93e9-9d2ebb30f415 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:04.362647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1088/2174] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:29:09.558245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c40d6b-bf09-4f87-8309-f1ea1dd72e62 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:09.560547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-06c40d6b-bf09-4f87-8309-f1ea1dd72e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:09.580988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c40d6b-bf09-4f87-8309-f1ea1dd72e62 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:09.592877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:09.595195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:09.599380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:09.599800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] PATH_INFO: `/v3/users/c349adc1e5054da7b96892d85b9bade7` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:09.604387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c349adc1e5054da7b96892d85b9bade7)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:09.606422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:09.710396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c349adc1e5054da7b96892d85b9bade7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:09.733031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c349adc1e5054da7b96892d85b9bade7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:09.733685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c349adc1e5054da7b96892d85b9bade7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:09.734391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c349adc1e5054da7b96892d85b9bade7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:09.748131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f9365b1-a455-4e64-ab5a-8b5fb93ce332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c349adc1e5054da7b96892d85b9bade7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:09.788042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1087/2175] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:29:09 2022] DELETE /identity/v3/users/c349adc1e5054da7b96892d85b9bade7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:09.793054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8f60f-60da-4993-bec5-e37e2ad22f07 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:09.793540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8f60f-60da-4993-bec5-e37e2ad22f07 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:09.793807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8f60f-60da-4993-bec5-e37e2ad22f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:09.839131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a1c8f60f-60da-4993-bec5-e37e2ad22f07 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:09.840652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a1c8f60f-60da-4993-bec5-e37e2ad22f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:09.887484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a8438-a783-494f-8c7e-0e642afaf2c9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:09.887989 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a8438-a783-494f-8c7e-0e642afaf2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:09.888363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a8438-a783-494f-8c7e-0e642afaf2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:09.927900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-425a8438-a783-494f-8c7e-0e642afaf2c9 None None] MFA Rules not processed for user `6073b9546a834ac4bb8801e7ed6eb4db`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:09.929227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-425a8438-a783-494f-8c7e-0e642afaf2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:09.978667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1089/2176] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:10.002060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8f60f-60da-4993-bec5-e37e2ad22f07 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:10.004187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a1c8f60f-60da-4993-bec5-e37e2ad22f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:10.034337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1088/2177] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:10.036506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a8438-a783-494f-8c7e-0e642afaf2c9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:10.038911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-425a8438-a783-494f-8c7e-0e642afaf2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:10.077330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a8438-a783-494f-8c7e-0e642afaf2c9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:10.090883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b8382-35e3-4a26-86ba-dd76d4346048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:10.096734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b8382-35e3-4a26-86ba-dd76d4346048 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:10.097554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b8382-35e3-4a26-86ba-dd76d4346048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:10.100287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b8382-35e3-4a26-86ba-dd76d4346048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:10.102304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b8382-35e3-4a26-86ba-dd76d4346048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:10.107767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b8382-35e3-4a26-86ba-dd76d4346048 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:10.150150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:10.152367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8f60f-60da-4993-bec5-e37e2ad22f07 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:10.163054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173d72b-dda3-4ba3-bd64-56fec4a58f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:10.166743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173d72b-dda3-4ba3-bd64-56fec4a58f7f service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:10.167134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173d72b-dda3-4ba3-bd64-56fec4a58f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:10.167320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173d72b-dda3-4ba3-bd64-56fec4a58f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:10.177138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1089/2178] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:29:10 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:29:10.184012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b8382-35e3-4a26-86ba-dd76d4346048 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:10.220019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:10.235529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173d72b-dda3-4ba3-bd64-56fec4a58f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:10.251756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173d72b-dda3-4ba3-bd64-56fec4a58f7f service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:10.295848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1090/2179] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:29:10.381111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b8382-35e3-4a26-86ba-dd76d4346048 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:10.388098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a0aba-f3a8-4feb-b1e9-c17369bcd735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:10.388995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a0aba-f3a8-4feb-b1e9-c17369bcd735 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:10.389440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a0aba-f3a8-4feb-b1e9-c17369bcd735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:10.389884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a0aba-f3a8-4feb-b1e9-c17369bcd735 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:10.390657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a0aba-f3a8-4feb-b1e9-c17369bcd735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:10.392756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a0aba-f3a8-4feb-b1e9-c17369bcd735 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:10.426391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1090/2180] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:29:10 2022] POST /identity/v3/users => generated 299 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:29:10.430805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173d72b-dda3-4ba3-bd64-56fec4a58f7f service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:10.448455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173d72b-dda3-4ba3-bd64-56fec4a58f7f service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:10.455471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77c4b4-09f4-4209-a96e-4532bff4d6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:10.456249 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77c4b4-09f4-4209-a96e-4532bff4d6fe admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:10.456613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77c4b4-09f4-4209-a96e-4532bff4d6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:10.457728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77c4b4-09f4-4209-a96e-4532bff4d6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:10.458785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77c4b4-09f4-4209-a96e-4532bff4d6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:10.462800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77c4b4-09f4-4209-a96e-4532bff4d6fe admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:10.479358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1091/2181] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:29:10.480655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a0aba-f3a8-4feb-b1e9-c17369bcd735 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:10.493165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a0aba-f3a8-4feb-b1e9-c17369bcd735 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:10.498696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574a00d-a1e8-4efc-8efb-8a9604b9ce30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:10.499746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574a00d-a1e8-4efc-8efb-8a9604b9ce30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:10.500038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574a00d-a1e8-4efc-8efb-8a9604b9ce30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:10.500315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574a00d-a1e8-4efc-8efb-8a9604b9ce30 admin admin] PATH_INFO: `/v3/projects/db66bf4fdc61433594f173b1f92fab39/users/34329bbfe65941a8b37b999622d8d9d4/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:10.525405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574a00d-a1e8-4efc-8efb-8a9604b9ce30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db66bf4fdc61433594f173b1f92fab39, user_id=34329bbfe65941a8b37b999622d8d9d4, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:10.528328 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574a00d-a1e8-4efc-8efb-8a9604b9ce30 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:10.557990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1091/2182] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:29:10 2022] PUT /identity/v3/projects/db66bf4fdc61433594f173b1f92fab39/users/34329bbfe65941a8b37b999622d8d9d4/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:10.563103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77c4b4-09f4-4209-a96e-4532bff4d6fe admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:10.575267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77c4b4-09f4-4209-a96e-4532bff4d6fe admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:10.678530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a62e5-e76b-4fe0-ad2e-9a5e15c9d1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:10.679242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a62e5-e76b-4fe0-ad2e-9a5e15c9d1df admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:10.679827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a62e5-e76b-4fe0-ad2e-9a5e15c9d1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:10.680170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a62e5-e76b-4fe0-ad2e-9a5e15c9d1df admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:10.682888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a62e5-e76b-4fe0-ad2e-9a5e15c9d1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:10.684647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a62e5-e76b-4fe0-ad2e-9a5e15c9d1df admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:10.714470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1092/2183] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:29:10.716593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574a00d-a1e8-4efc-8efb-8a9604b9ce30 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:10.728044 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574a00d-a1e8-4efc-8efb-8a9604b9ce30 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:10.733517 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eca025-1f17-44d0-9d8f-96b5ca41d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:10.734315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eca025-1f17-44d0-9d8f-96b5ca41d5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:10.735226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eca025-1f17-44d0-9d8f-96b5ca41d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:10.735572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eca025-1f17-44d0-9d8f-96b5ca41d5cf admin admin] PATH_INFO: `/v3/projects/db66bf4fdc61433594f173b1f92fab39/users/34329bbfe65941a8b37b999622d8d9d4/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:10.737846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eca025-1f17-44d0-9d8f-96b5ca41d5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=db66bf4fdc61433594f173b1f92fab39, user_id=34329bbfe65941a8b37b999622d8d9d4, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:10.740990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eca025-1f17-44d0-9d8f-96b5ca41d5cf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:10.758517 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1092/2184] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:29:10 2022] PUT /identity/v3/projects/db66bf4fdc61433594f173b1f92fab39/users/34329bbfe65941a8b37b999622d8d9d4/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:29:10.866406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a62e5-e76b-4fe0-ad2e-9a5e15c9d1df admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:10.869415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6a1a62e5-e76b-4fe0-ad2e-9a5e15c9d1df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:10.886200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a62e5-e76b-4fe0-ad2e-9a5e15c9d1df admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:11.063451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:11.064122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.064401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.064681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] PATH_INFO: `/v3/projects/35c6a3aec31d419488ba024ea687ff19` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.065917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c6a3aec31d419488ba024ea687ff19)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:11.068255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:11.156114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c6a3aec31d419488ba024ea687ff19', 'request_id': 'req-2f5e95b2-7e29-418b-b40b-38caeb417af0'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:11.163219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c6a3aec31d419488ba024ea687ff19', 'request_id': 'req-2f5e95b2-7e29-418b-b40b-38caeb417af0'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:11.174258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c6a3aec31d419488ba024ea687ff19', 'request_id': 'req-2f5e95b2-7e29-418b-b40b-38caeb417af0'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:11.193026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] The token cache is being invalidate because project 35c6a3aec31d419488ba024ea687ff19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:29:11.193372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:11.194104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2f5e95b2-7e29-418b-b40b-38caeb417af0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:11.196087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1093/2185] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:29:10 2022] DELETE /identity/v3/projects/35c6a3aec31d419488ba024ea687ff19 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:11.201311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a4411-813a-45ff-bba5-bb2dd4ac4340 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.201311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a4411-813a-45ff-bba5-bb2dd4ac4340 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.201556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a4411-813a-45ff-bba5-bb2dd4ac4340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.224840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1f1a4411-813a-45ff-bba5-bb2dd4ac4340 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:11.226060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1f1a4411-813a-45ff-bba5-bb2dd4ac4340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:11.329539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1093/2186] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:11.337369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628deb-1e01-4f2b-b812-6b887e80faf7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.337658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628deb-1e01-4f2b-b812-6b887e80faf7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.337956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628deb-1e01-4f2b-b812-6b887e80faf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.375203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-31628deb-1e01-4f2b-b812-6b887e80faf7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:11.376718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-31628deb-1e01-4f2b-b812-6b887e80faf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:11.397123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1094/2187] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:11.401815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a4411-813a-45ff-bba5-bb2dd4ac4340 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:11.422170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a4411-813a-45ff-bba5-bb2dd4ac4340 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:11.427923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9e09b-e9a9-42af-8a6e-cd7bba8610be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:11.428713 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9e09b-e9a9-42af-8a6e-cd7bba8610be admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.429046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9e09b-e9a9-42af-8a6e-cd7bba8610be admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.429334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9e09b-e9a9-42af-8a6e-cd7bba8610be admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.430677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9e09b-e9a9-42af-8a6e-cd7bba8610be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:29:11.430980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9e09b-e9a9-42af-8a6e-cd7bba8610be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:11.432666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9e09b-e9a9-42af-8a6e-cd7bba8610be admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:11.445972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1094/2188] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:29:11.449662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31628deb-1e01-4f2b-b812-6b887e80faf7 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:11.454475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:11.461731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31628deb-1e01-4f2b-b812-6b887e80faf7 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:11.469025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc00e3-b41a-40a6-afb8-1aabb9bd754d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:11.469617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc00e3-b41a-40a6-afb8-1aabb9bd754d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.470312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc00e3-b41a-40a6-afb8-1aabb9bd754d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.470312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc00e3-b41a-40a6-afb8-1aabb9bd754d admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.470898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc00e3-b41a-40a6-afb8-1aabb9bd754d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:11.473043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc00e3-b41a-40a6-afb8-1aabb9bd754d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:11.506731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1095/2189] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:29:11 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:29:11.510942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9e09b-e9a9-42af-8a6e-cd7bba8610be admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:11.521642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9e09b-e9a9-42af-8a6e-cd7bba8610be admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:11.559776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:11.609970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79a823-091d-42b2-b316-492837aa8d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:11.615871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79a823-091d-42b2-b316-492837aa8d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.616715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79a823-091d-42b2-b316-492837aa8d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.617232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79a823-091d-42b2-b316-492837aa8d15 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.618166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79a823-091d-42b2-b316-492837aa8d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:11.620464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79a823-091d-42b2-b316-492837aa8d15 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:11.654720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1095/2190] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:29:11 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:29:11.655947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc00e3-b41a-40a6-afb8-1aabb9bd754d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:11.677382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc00e3-b41a-40a6-afb8-1aabb9bd754d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:11.682691 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89411be9-2686-4f54-8e76-78d6cfde22ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:11.683324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89411be9-2686-4f54-8e76-78d6cfde22ba admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.683601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89411be9-2686-4f54-8e76-78d6cfde22ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.683872 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89411be9-2686-4f54-8e76-78d6cfde22ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.684513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89411be9-2686-4f54-8e76-78d6cfde22ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:11.686538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89411be9-2686-4f54-8e76-78d6cfde22ba admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:11.701896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1096/2191] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:29:11.703009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79a823-091d-42b2-b316-492837aa8d15 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:11.741844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79a823-091d-42b2-b316-492837aa8d15 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:11.754867 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef32bb-67b9-49c0-a778-76ad457515ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:11.755594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef32bb-67b9-49c0-a778-76ad457515ca admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.755932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef32bb-67b9-49c0-a778-76ad457515ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.756485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef32bb-67b9-49c0-a778-76ad457515ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.757362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef32bb-67b9-49c0-a778-76ad457515ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:11.760841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef32bb-67b9-49c0-a778-76ad457515ca admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:11.795333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1096/2192] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:29:11.799545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89411be9-2686-4f54-8e76-78d6cfde22ba admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:11.814068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89411be9-2686-4f54-8e76-78d6cfde22ba admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:11.835137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec9ae-458b-4b69-a10a-976dc3e6fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:11.836488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec9ae-458b-4b69-a10a-976dc3e6fa45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.836835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec9ae-458b-4b69-a10a-976dc3e6fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.837207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec9ae-458b-4b69-a10a-976dc3e6fa45 admin admin] PATH_INFO: `/v3/projects/6deb1edc63ed425eafbea3dc1dded3ac/users/07df6c0689084d3d95338d3fe0627b1b/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.874461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fec9ae-458b-4b69-a10a-976dc3e6fa45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6deb1edc63ed425eafbea3dc1dded3ac, user_id=07df6c0689084d3d95338d3fe0627b1b, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:11.877459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fec9ae-458b-4b69-a10a-976dc3e6fa45 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:11.904225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1097/2193] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:29:11 2022] PUT /identity/v3/projects/6deb1edc63ed425eafbea3dc1dded3ac/users/07df6c0689084d3d95338d3fe0627b1b/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:11.909313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f0e99-78a4-4081-976c-713b2ae6f53d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:11.909600 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f0e99-78a4-4081-976c-713b2ae6f53d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:11.909877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f0e99-78a4-4081-976c-713b2ae6f53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:11.951578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-bc6f0e99-78a4-4081-976c-713b2ae6f53d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:11.955524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bc6f0e99-78a4-4081-976c-713b2ae6f53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:12.046426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1097/2194] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:12.055356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec9ae-458b-4b69-a10a-976dc3e6fa45 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:12.057150 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e7fec9ae-458b-4b69-a10a-976dc3e6fa45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:12.134732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec9ae-458b-4b69-a10a-976dc3e6fa45 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:12.141768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d8f36-dde2-490e-a184-b41be7cf9945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:12.142690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d8f36-dde2-490e-a184-b41be7cf9945 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:12.143162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d8f36-dde2-490e-a184-b41be7cf9945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:12.143755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d8f36-dde2-490e-a184-b41be7cf9945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:12.157878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d8f36-dde2-490e-a184-b41be7cf9945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:12.159878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d8f36-dde2-490e-a184-b41be7cf9945 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:12.175399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1098/2195] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:29:13.732681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b3fe5-f099-4074-b2a2-1eda22b2c860 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:13.733389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b3fe5-f099-4074-b2a2-1eda22b2c860 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:13.733389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b3fe5-f099-4074-b2a2-1eda22b2c860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:13.815402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5e5b3fe5-f099-4074-b2a2-1eda22b2c860 None None] MFA Rules not processed for user `07df6c0689084d3d95338d3fe0627b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:13.816859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5e5b3fe5-f099-4074-b2a2-1eda22b2c860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:13.929067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1098/2196] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:13.941580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d8f36-dde2-490e-a184-b41be7cf9945 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:13.943259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-479d8f36-dde2-490e-a184-b41be7cf9945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:14.058916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d8f36-dde2-490e-a184-b41be7cf9945 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:14.072682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49790db-dc65-462a-b54a-f97cce508197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:14.078224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49790db-dc65-462a-b54a-f97cce508197 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:14.078573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49790db-dc65-462a-b54a-f97cce508197 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:14.078853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49790db-dc65-462a-b54a-f97cce508197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:14.127533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49790db-dc65-462a-b54a-f97cce508197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:14.129014 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49790db-dc65-462a-b54a-f97cce508197 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:14.178356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1099/2197] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:29:16.010058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71364795-001b-4c7e-bfb3-5858c664c0f7 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:16.011019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71364795-001b-4c7e-bfb3-5858c664c0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:16.011512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71364795-001b-4c7e-bfb3-5858c664c0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:16.034969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-71364795-001b-4c7e-bfb3-5858c664c0f7 None None] MFA Rules not processed for user `07df6c0689084d3d95338d3fe0627b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:16.036445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-71364795-001b-4c7e-bfb3-5858c664c0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:16.045325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1099/2198] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:19.359270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01b692-63af-409a-a66d-8219c53fd0b9 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:19.359764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01b692-63af-409a-a66d-8219c53fd0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:19.359876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01b692-63af-409a-a66d-8219c53fd0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:19.407964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7f01b692-63af-409a-a66d-8219c53fd0b9 None None] MFA Rules not processed for user `34329bbfe65941a8b37b999622d8d9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:19.409214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7f01b692-63af-409a-a66d-8219c53fd0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:19.460014 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:19.495092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1100/2199] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:29:19.508438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71364795-001b-4c7e-bfb3-5858c664c0f7 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:19.510623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-71364795-001b-4c7e-bfb3-5858c664c0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:19.560518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:19.579183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71364795-001b-4c7e-bfb3-5858c664c0f7 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:19.583695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5305f8a5d2b540de91d3caedad454bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:19.584575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:19.585033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:19.585474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:19.588083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:19.660966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:19.662552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:19.672449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1100/2200] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:29:19.869546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163b5eb-03be-4bb8-8c7f-54d762ea2f7a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:19.869833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163b5eb-03be-4bb8-8c7f-54d762ea2f7a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:19.870143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163b5eb-03be-4bb8-8c7f-54d762ea2f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:19.890249 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a163b5eb-03be-4bb8-8c7f-54d762ea2f7a None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:19.891452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a163b5eb-03be-4bb8-8c7f-54d762ea2f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:19.948890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1101/2201] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:19.990133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:19.991525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:20.059831 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6459e62-b31f-462f-975a-e7768911cf12 service glance] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:20.064671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da98c8-cce1-4508-a586-e770aaf62325 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:20.065277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da98c8-cce1-4508-a586-e770aaf62325 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:20.065555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da98c8-cce1-4508-a586-e770aaf62325 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:20.065851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da98c8-cce1-4508-a586-e770aaf62325 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:20.075072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da98c8-cce1-4508-a586-e770aaf62325 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:20.076647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da98c8-cce1-4508-a586-e770aaf62325 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:20.085766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1101/2202] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:29:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:29:24.551863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ccb77-7fbb-4695-974f-5849ea000762 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:24.552392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ccb77-7fbb-4695-974f-5849ea000762 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:24.552483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ccb77-7fbb-4695-974f-5849ea000762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:24.611164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a88ccb77-7fbb-4695-974f-5849ea000762 None None] MFA Rules not processed for user `c6768e1bc2a7478c8c662a9c298d72e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:24.616263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a88ccb77-7fbb-4695-974f-5849ea000762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:24.819406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1102/2203] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:28.589789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da98c8-cce1-4508-a586-e770aaf62325 service swift] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:28.596347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-38da98c8-cce1-4508-a586-e770aaf62325 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:28.615879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da98c8-cce1-4508-a586-e770aaf62325 service swift] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:28.622769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:28.623721 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:28.624206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:28.624675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] PATH_INFO: `/v3/users/29f82f899fa64b8eafaf966e6a83f600` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:28.627281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f82f899fa64b8eafaf966e6a83f600)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:28.629304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:28.719210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f82f899fa64b8eafaf966e6a83f600'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:28.743273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f82f899fa64b8eafaf966e6a83f600'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:28.744192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f82f899fa64b8eafaf966e6a83f600'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:28.744955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f82f899fa64b8eafaf966e6a83f600'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:28.773420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f82f899fa64b8eafaf966e6a83f600'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:28.794585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:28.817561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1102/2204] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:29:28 2022] DELETE /identity/v3/users/29f82f899fa64b8eafaf966e6a83f600 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:28.819016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593f4e2-2a05-4401-8dd7-a7626f49a413 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:28.819528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593f4e2-2a05-4401-8dd7-a7626f49a413 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:28.820218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593f4e2-2a05-4401-8dd7-a7626f49a413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:28.861153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f593f4e2-2a05-4401-8dd7-a7626f49a413 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:28.862689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f593f4e2-2a05-4401-8dd7-a7626f49a413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:29.017810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1103/2205] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:29:29.039868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:29.044583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:29.137100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded54a5-05e2-489c-b345-77dd3af45ac4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:29.141245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a44791-c175-4a93-8636-ed67f0961b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:29.142172 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a44791-c175-4a93-8636-ed67f0961b8e service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:29.142658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a44791-c175-4a93-8636-ed67f0961b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:29.143281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a44791-c175-4a93-8636-ed67f0961b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:29.181990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a44791-c175-4a93-8636-ed67f0961b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:29.183624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a44791-c175-4a93-8636-ed67f0961b8e service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:29.221212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1103/2206] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:29:30.131172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593f4e2-2a05-4401-8dd7-a7626f49a413 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:30.136049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f593f4e2-2a05-4401-8dd7-a7626f49a413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:30.160590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593f4e2-2a05-4401-8dd7-a7626f49a413 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:30.189731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:30.190833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:30.191360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:30.191857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] PATH_INFO: `/v3/projects/3131c1978b9748e1bee2fa8a974bfdeb` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:30.193530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3131c1978b9748e1bee2fa8a974bfdeb)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:30.196402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:30.318799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:30.401727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3131c1978b9748e1bee2fa8a974bfdeb', 'request_id': 'req-1b68d567-b764-43ee-a131-df4102627357'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:30.408992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3131c1978b9748e1bee2fa8a974bfdeb', 'request_id': 'req-1b68d567-b764-43ee-a131-df4102627357'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:30.422288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3131c1978b9748e1bee2fa8a974bfdeb', 'request_id': 'req-1b68d567-b764-43ee-a131-df4102627357'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:30.436881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] The token cache is being invalidate because project 3131c1978b9748e1bee2fa8a974bfdeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:29:30.437228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:30.437852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1b68d567-b764-43ee-a131-df4102627357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:30.439870 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1104/2207] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:29:30 2022] DELETE /identity/v3/projects/3131c1978b9748e1bee2fa8a974bfdeb => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:30.444502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dedcff-89af-4c56-840f-9a334eea80bd None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:30.445403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dedcff-89af-4c56-840f-9a334eea80bd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:30.445838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dedcff-89af-4c56-840f-9a334eea80bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:30.499764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b3dedcff-89af-4c56-840f-9a334eea80bd None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:30.500855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b3dedcff-89af-4c56-840f-9a334eea80bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:30.716744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1104/2208] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:30.719388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c65511-3651-4f6e-abd4-14fad27250b5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:30.719679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c65511-3651-4f6e-abd4-14fad27250b5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:30.719953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c65511-3651-4f6e-abd4-14fad27250b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:30.746572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-59c65511-3651-4f6e-abd4-14fad27250b5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:30.748006 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-59c65511-3651-4f6e-abd4-14fad27250b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:30.759453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1105/2209] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:30.763590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dedcff-89af-4c56-840f-9a334eea80bd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:30.780037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dedcff-89af-4c56-840f-9a334eea80bd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:30.786446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93fbbc-10a8-42f5-acbc-e3081073aa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:30.788178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93fbbc-10a8-42f5-acbc-e3081073aa10 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:30.788554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93fbbc-10a8-42f5-acbc-e3081073aa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:30.788847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93fbbc-10a8-42f5-acbc-e3081073aa10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:30.789524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93fbbc-10a8-42f5-acbc-e3081073aa10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:29:30.789834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93fbbc-10a8-42f5-acbc-e3081073aa10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:30.822497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93fbbc-10a8-42f5-acbc-e3081073aa10 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:30.854179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1105/2210] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:29:30.863092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c65511-3651-4f6e-abd4-14fad27250b5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:30.873252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c65511-3651-4f6e-abd4-14fad27250b5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:30.878806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79aca2-51dc-4f8c-b06c-668fbff251fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:30.879502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79aca2-51dc-4f8c-b06c-668fbff251fa admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:30.879858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79aca2-51dc-4f8c-b06c-668fbff251fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:30.880201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79aca2-51dc-4f8c-b06c-668fbff251fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:30.881076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79aca2-51dc-4f8c-b06c-668fbff251fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:30.884657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79aca2-51dc-4f8c-b06c-668fbff251fa admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:30.944448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1106/2211] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:29:30 2022] POST /identity/v3/projects => generated 391 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:29:30.950850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93fbbc-10a8-42f5-acbc-e3081073aa10 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:30.964068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93fbbc-10a8-42f5-acbc-e3081073aa10 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:31.132745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8bb83-8df6-418a-bd4f-f48c49076845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:31.133519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8bb83-8df6-418a-bd4f-f48c49076845 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:31.134367 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8bb83-8df6-418a-bd4f-f48c49076845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:31.134660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8bb83-8df6-418a-bd4f-f48c49076845 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:31.136572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8bb83-8df6-418a-bd4f-f48c49076845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:31.139782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8bb83-8df6-418a-bd4f-f48c49076845 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:31.222896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1106/2212] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:29:30 2022] POST /identity/v3/users => generated 296 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:29:31.230668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79aca2-51dc-4f8c-b06c-668fbff251fa admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:31.250778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79aca2-51dc-4f8c-b06c-668fbff251fa admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:31.258392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb92ac-0c31-41b0-9d96-2ce0b3d567d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:31.259658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb92ac-0c31-41b0-9d96-2ce0b3d567d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:31.260885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb92ac-0c31-41b0-9d96-2ce0b3d567d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:31.261379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb92ac-0c31-41b0-9d96-2ce0b3d567d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:31.262800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb92ac-0c31-41b0-9d96-2ce0b3d567d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:31.264822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb92ac-0c31-41b0-9d96-2ce0b3d567d7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:31.280724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1107/2213] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:29:31.285224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8bb83-8df6-418a-bd4f-f48c49076845 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:31.299770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8bb83-8df6-418a-bd4f-f48c49076845 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:31.326626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131501-0fd9-4b00-a4fd-eac7d53186a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:31.327430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131501-0fd9-4b00-a4fd-eac7d53186a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:31.334205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131501-0fd9-4b00-a4fd-eac7d53186a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:31.334498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131501-0fd9-4b00-a4fd-eac7d53186a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:31.335225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131501-0fd9-4b00-a4fd-eac7d53186a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:31.336859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131501-0fd9-4b00-a4fd-eac7d53186a5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:31.363773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1107/2214] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:29:31.368407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb92ac-0c31-41b0-9d96-2ce0b3d567d7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:31.383155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb92ac-0c31-41b0-9d96-2ce0b3d567d7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:31.398636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1ac77-1e23-40e3-bf1e-6124eee9a0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:31.399755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1ac77-1e23-40e3-bf1e-6124eee9a0fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:31.400262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1ac77-1e23-40e3-bf1e-6124eee9a0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:31.400759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1ac77-1e23-40e3-bf1e-6124eee9a0fa admin admin] PATH_INFO: `/v3/projects/a25192b5cd2b4b388552ab6a38647bf5/users/00d7a0fccfce4439994abf4759151b62/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:31.438578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1ac77-1e23-40e3-bf1e-6124eee9a0fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25192b5cd2b4b388552ab6a38647bf5, user_id=00d7a0fccfce4439994abf4759151b62, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:31.441268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1ac77-1e23-40e3-bf1e-6124eee9a0fa admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:31.514189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1108/2215] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:29:31 2022] PUT /identity/v3/projects/a25192b5cd2b4b388552ab6a38647bf5/users/00d7a0fccfce4439994abf4759151b62/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:31.526569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954ccc3-93ad-4eff-9b80-cf5b338ae160 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:31.527007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954ccc3-93ad-4eff-9b80-cf5b338ae160 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:31.527007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954ccc3-93ad-4eff-9b80-cf5b338ae160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:31.551488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:31.602840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3954ccc3-93ad-4eff-9b80-cf5b338ae160 None None] MFA Rules not processed for user `00d7a0fccfce4439994abf4759151b62`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:31.604120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3954ccc3-93ad-4eff-9b80-cf5b338ae160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:31.695578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1108/2216] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:31.702990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1ac77-1e23-40e3-bf1e-6124eee9a0fa admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:31.705801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3be1ac77-1e23-40e3-bf1e-6124eee9a0fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:31.779394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1ac77-1e23-40e3-bf1e-6124eee9a0fa admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:31.800630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c7cfd-0cf3-476b-9612-4d77238ba0d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:31.801599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c7cfd-0cf3-476b-9612-4d77238ba0d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:31.802392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c7cfd-0cf3-476b-9612-4d77238ba0d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:31.802699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c7cfd-0cf3-476b-9612-4d77238ba0d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:31.813764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c7cfd-0cf3-476b-9612-4d77238ba0d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:31.815435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c7cfd-0cf3-476b-9612-4d77238ba0d6 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:31.827971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1109/2217] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:29:34.200783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56fd85-3131-4fc5-bd48-5994ea81ab57 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:34.201319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56fd85-3131-4fc5-bd48-5994ea81ab57 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:34.201319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56fd85-3131-4fc5-bd48-5994ea81ab57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:34.221581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8a56fd85-3131-4fc5-bd48-5994ea81ab57 None None] MFA Rules not processed for user `1ec24729b04947b7b05cc6dcd787eb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:34.222895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a56fd85-3131-4fc5-bd48-5994ea81ab57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:34.290364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1109/2218] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:36.065065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab501d4-2091-4618-84d8-7d41e564dbfd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:36.065573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab501d4-2091-4618-84d8-7d41e564dbfd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:36.065666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab501d4-2091-4618-84d8-7d41e564dbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:36.084504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9ab501d4-2091-4618-84d8-7d41e564dbfd None None] MFA Rules not processed for user `6073b9546a834ac4bb8801e7ed6eb4db`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:36.085743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9ab501d4-2091-4618-84d8-7d41e564dbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:36.130766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:36.144829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1110/2219] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:44.281166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=56 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:46.142851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:50.938112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e68a8-8b1b-40c2-a600-876d76960e90 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:50.939038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e68a8-8b1b-40c2-a600-876d76960e90 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:50.940246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e68a8-8b1b-40c2-a600-876d76960e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:50.977063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5c4e68a8-8b1b-40c2-a600-876d76960e90 None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:50.978686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5c4e68a8-8b1b-40c2-a600-876d76960e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:51.052532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1110/2220] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:29:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:51.066678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab501d4-2091-4618-84d8-7d41e564dbfd None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:51.069064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9ab501d4-2091-4618-84d8-7d41e564dbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:51.152199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab501d4-2091-4618-84d8-7d41e564dbfd None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:51.158521 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af090d8c-ba53-411c-bbf1-cc65c9615f95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:51.159374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af090d8c-ba53-411c-bbf1-cc65c9615f95 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:51.159658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af090d8c-ba53-411c-bbf1-cc65c9615f95 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:51.159982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af090d8c-ba53-411c-bbf1-cc65c9615f95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:51.171868 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af090d8c-ba53-411c-bbf1-cc65c9615f95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:51.173875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af090d8c-ba53-411c-bbf1-cc65c9615f95 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:51.184591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1111/2221] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:29:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:29:52.079782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e68a8-8b1b-40c2-a600-876d76960e90 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:52.171286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e68a8-8b1b-40c2-a600-876d76960e90 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:52.175695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:52.176329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:52.176604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:52.176874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] PATH_INFO: `/v3/users/00d7a0fccfce4439994abf4759151b62` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:52.178135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d7a0fccfce4439994abf4759151b62)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:52.179990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:52.238750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d7a0fccfce4439994abf4759151b62'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:52.267239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d7a0fccfce4439994abf4759151b62'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:52.269145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d7a0fccfce4439994abf4759151b62'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:52.269956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d7a0fccfce4439994abf4759151b62'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:52.281394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d7a0fccfce4439994abf4759151b62'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:52.314868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1111/2222] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:29:52 2022] DELETE /identity/v3/users/00d7a0fccfce4439994abf4759151b62 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:29:52.319527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc8dc2-bbc3-4af5-908c-2167168c85f8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:52.320014 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc8dc2-bbc3-4af5-908c-2167168c85f8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:52.320323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc8dc2-bbc3-4af5-908c-2167168c85f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:52.370165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6dcc8dc2-bbc3-4af5-908c-2167168c85f8 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:29:52.370165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6dcc8dc2-bbc3-4af5-908c-2167168c85f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:52.503291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1112/2223] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:29:52.518647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:52.520578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:52.556405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:29:52.594565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a496e9-f32b-46ba-b7b1-b9ca4f9689d0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:52.601486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ae0f4-5dac-4088-b690-7cb554ef0a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:52.602270 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ae0f4-5dac-4088-b690-7cb554ef0a6e service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:52.602633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ae0f4-5dac-4088-b690-7cb554ef0a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:52.602979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ae0f4-5dac-4088-b690-7cb554ef0a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:52.613109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ae0f4-5dac-4088-b690-7cb554ef0a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:52.614796 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ae0f4-5dac-4088-b690-7cb554ef0a6e service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:52.626016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1112/2224] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:29:53.061741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc8dc2-bbc3-4af5-908c-2167168c85f8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:29:53.071220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6dcc8dc2-bbc3-4af5-908c-2167168c85f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:29:53.086903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc8dc2-bbc3-4af5-908c-2167168c85f8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:29:53.097105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:29:53.097831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:29:53.098143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:29:53.098418 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] PATH_INFO: `/v3/projects/a25192b5cd2b4b388552ab6a38647bf5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:29:53.099701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25192b5cd2b4b388552ab6a38647bf5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:29:53.104841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:29:53.201040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25192b5cd2b4b388552ab6a38647bf5', 'request_id': 'req-d29d57c4-c5a2-4716-91bc-2de6b92c8635'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:53.211835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25192b5cd2b4b388552ab6a38647bf5', 'request_id': 'req-d29d57c4-c5a2-4716-91bc-2de6b92c8635'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:53.224617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25192b5cd2b4b388552ab6a38647bf5', 'request_id': 'req-d29d57c4-c5a2-4716-91bc-2de6b92c8635'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:53.241078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] The token cache is being invalidate because project a25192b5cd2b4b388552ab6a38647bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:29:53.241375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:53.242108 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d29d57c4-c5a2-4716-91bc-2de6b92c8635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:29:53.251586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1113/2225] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:29:53 2022] DELETE /identity/v3/projects/a25192b5cd2b4b388552ab6a38647bf5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:30:02.620462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=15 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:30:03.238084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:30:12.027077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9e831-f3ed-4bc9-9883-b7f997542410 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:12.027573 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9e831-f3ed-4bc9-9883-b7f997542410 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:12.027668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9e831-f3ed-4bc9-9883-b7f997542410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:12.046585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b6a9e831-f3ed-4bc9-9883-b7f997542410 None None] MFA Rules not processed for user `e7527106829643448f153f9431f233f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:12.047786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b6a9e831-f3ed-4bc9-9883-b7f997542410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:12.102691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1113/2226] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:30:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:17.787978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d305c0-b2be-4ff9-9894-ed8ade9c95f1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:17.789828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d305c0-b2be-4ff9-9894-ed8ade9c95f1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:17.790331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d305c0-b2be-4ff9-9894-ed8ade9c95f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:17.811049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-67d305c0-b2be-4ff9-9894-ed8ade9c95f1 None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:17.812242 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-67d305c0-b2be-4ff9-9894-ed8ade9c95f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:17.867802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1114/2227] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:30:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:17.881633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9e831-f3ed-4bc9-9883-b7f997542410 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:17.883674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b6a9e831-f3ed-4bc9-9883-b7f997542410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:17.959608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9e831-f3ed-4bc9-9883-b7f997542410 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:17.974330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243c45b-7dd0-4cb7-8b2a-366f6db89ff8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:17.974330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243c45b-7dd0-4cb7-8b2a-366f6db89ff8 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:17.974330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243c45b-7dd0-4cb7-8b2a-366f6db89ff8 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:17.974330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243c45b-7dd0-4cb7-8b2a-366f6db89ff8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:17.985568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243c45b-7dd0-4cb7-8b2a-366f6db89ff8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:17.987681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243c45b-7dd0-4cb7-8b2a-366f6db89ff8 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:17.998819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1114/2228] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:30:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:30:27.872395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=25 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:30:27.993432 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=57 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:30:32.140863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eafd3e-30eb-4c2e-a20a-7c035bf9ce9e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:32.141472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eafd3e-30eb-4c2e-a20a-7c035bf9ce9e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:32.141472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eafd3e-30eb-4c2e-a20a-7c035bf9ce9e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:32.142820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1115/2229] 10.222.0.18 () {54 vars in 961 bytes} [Wed Sep 21 05:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:30:32.153083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af96745-6132-494d-b5c5-b9a31247be9d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:32.154033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af96745-6132-494d-b5c5-b9a31247be9d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:32.154612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af96745-6132-494d-b5c5-b9a31247be9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:32.183285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0af96745-6132-494d-b5c5-b9a31247be9d None None] MFA Rules not processed for user `4b3d8a1c8b6640ae9e90ece0bbfbf410`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:32.184779 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0af96745-6132-494d-b5c5-b9a31247be9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:32.373132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1115/2230] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Sep 21 05:30:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:32.387274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eafd3e-30eb-4c2e-a20a-7c035bf9ce9e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:32.413451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eafd3e-30eb-4c2e-a20a-7c035bf9ce9e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:32.432605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd4ca1-7c7c-44f5-9770-861065d26036 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:32.433238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd4ca1-7c7c-44f5-9770-861065d26036 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:32.433571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd4ca1-7c7c-44f5-9770-861065d26036 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:32.433795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd4ca1-7c7c-44f5-9770-861065d26036 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:32.439945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-13fd4ca1-7c7c-44f5-9770-861065d26036 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:32.459594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd4ca1-7c7c-44f5-9770-861065d26036 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:32.464058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd4ca1-7c7c-44f5-9770-861065d26036 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:32.505410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1116/2231] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:30:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:30:41.937826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af96745-6132-494d-b5c5-b9a31247be9d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:41.939307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0af96745-6132-494d-b5c5-b9a31247be9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:42.002431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af96745-6132-494d-b5c5-b9a31247be9d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:42.006295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb5db4-d069-4b2b-a74b-dc31e55402fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:42.006926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb5db4-d069-4b2b-a74b-dc31e55402fe service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:42.007247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb5db4-d069-4b2b-a74b-dc31e55402fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:42.007515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb5db4-d069-4b2b-a74b-dc31e55402fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:42.009277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-72eb5db4-d069-4b2b-a74b-dc31e55402fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:42.073544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb5db4-d069-4b2b-a74b-dc31e55402fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:42.075266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb5db4-d069-4b2b-a74b-dc31e55402fe service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:42.084837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1116/2232] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:30:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:30:42.376144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253b05-e1ee-4d14-b555-f86e7fd0d5d2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:42.376647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253b05-e1ee-4d14-b555-f86e7fd0d5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:42.378138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253b05-e1ee-4d14-b555-f86e7fd0d5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:42.418589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-06253b05-e1ee-4d14-b555-f86e7fd0d5d2 None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:42.420151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-06253b05-e1ee-4d14-b555-f86e7fd0d5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:42.429120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1117/2233] 10.222.0.18 () {58 vars in 967 bytes} [Wed Sep 21 05:30:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:42.444206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb5db4-d069-4b2b-a74b-dc31e55402fe service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:42.463033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb5db4-d069-4b2b-a74b-dc31e55402fe service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:42.473255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356603a-aa05-4712-9154-7002709ea9bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:42.474563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356603a-aa05-4712-9154-7002709ea9bc service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:42.475167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356603a-aa05-4712-9154-7002709ea9bc service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:42.476388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356603a-aa05-4712-9154-7002709ea9bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:42.498169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356603a-aa05-4712-9154-7002709ea9bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:42.500890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356603a-aa05-4712-9154-7002709ea9bc service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:42.511043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1117/2234] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:30:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:30:44.829254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a2d42-aabf-4848-9806-bbdbc4072c31 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:44.830280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a2d42-aabf-4848-9806-bbdbc4072c31 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:44.830834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a2d42-aabf-4848-9806-bbdbc4072c31 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:44.833101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1118/2235] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:30:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:30:44.864809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50867daa-0377-436e-ad56-36c9ff7ada1f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:44.865800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50867daa-0377-436e-ad56-36c9ff7ada1f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:44.866111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50867daa-0377-436e-ad56-36c9ff7ada1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:44.889250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-50867daa-0377-436e-ad56-36c9ff7ada1f None None] MFA Rules not processed for user `4b3d8a1c8b6640ae9e90ece0bbfbf410`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:44.890521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-50867daa-0377-436e-ad56-36c9ff7ada1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:44.900693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1118/2236] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:30:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:44.919282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a2d42-aabf-4848-9806-bbdbc4072c31 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:44.921587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d97a2d42-aabf-4848-9806-bbdbc4072c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:44.946218 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50867daa-0377-436e-ad56-36c9ff7ada1f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:44.948331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-50867daa-0377-436e-ad56-36c9ff7ada1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:44.955771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a2d42-aabf-4848-9806-bbdbc4072c31 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:44.979274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e6dd7-2a9f-4690-a3ad-d9a7c0a44d53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:44.980292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e6dd7-2a9f-4690-a3ad-d9a7c0a44d53 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:44.980292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e6dd7-2a9f-4690-a3ad-d9a7c0a44d53 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:44.980512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e6dd7-2a9f-4690-a3ad-d9a7c0a44d53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:45.014261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e6dd7-2a9f-4690-a3ad-d9a7c0a44d53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:45.016543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e6dd7-2a9f-4690-a3ad-d9a7c0a44d53 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:45.030128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1119/2237] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:30:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:30:45.051910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:30:45.119434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50867daa-0377-436e-ad56-36c9ff7ada1f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:45.124417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:45.125125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:45.125476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:45.125819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] PATH_INFO: `/v3/users/c6768e1bc2a7478c8c662a9c298d72e1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:45.127057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6768e1bc2a7478c8c662a9c298d72e1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:45.131146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:45.213960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6768e1bc2a7478c8c662a9c298d72e1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.228506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6768e1bc2a7478c8c662a9c298d72e1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.229262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6768e1bc2a7478c8c662a9c298d72e1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.230052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6768e1bc2a7478c8c662a9c298d72e1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.240773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e27906de-35bd-4424-a96e-b068efa9d875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6768e1bc2a7478c8c662a9c298d72e1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.275706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1119/2238] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:30:44 2022] DELETE /identity/v3/users/c6768e1bc2a7478c8c662a9c298d72e1 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:30:45.676556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e6dd7-2a9f-4690-a3ad-d9a7c0a44d53 service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:45.698300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e6dd7-2a9f-4690-a3ad-d9a7c0a44d53 service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:45.796217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:45.798179 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:45.808016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:45.809124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] PATH_INFO: `/v3/projects/eb6836d2a0994f27be0012f2608143f0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:45.811999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb6836d2a0994f27be0012f2608143f0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:45.816229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:45.908527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb6836d2a0994f27be0012f2608143f0', 'request_id': 'req-6610e5db-179a-4c42-90ba-bf1c71581402'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.918770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb6836d2a0994f27be0012f2608143f0', 'request_id': 'req-6610e5db-179a-4c42-90ba-bf1c71581402'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.935388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb6836d2a0994f27be0012f2608143f0', 'request_id': 'req-6610e5db-179a-4c42-90ba-bf1c71581402'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.967654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] The token cache is being invalidate because project eb6836d2a0994f27be0012f2608143f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:30:45.968570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.968833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6610e5db-179a-4c42-90ba-bf1c71581402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:30:45.972806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1120/2239] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:30:45 2022] DELETE /identity/v3/projects/eb6836d2a0994f27be0012f2608143f0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:30:45.978582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7c592-7dbd-4543-a2c8-8a350322b442 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:45.979458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7c592-7dbd-4543-a2c8-8a350322b442 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:45.979884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7c592-7dbd-4543-a2c8-8a350322b442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:46.023621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e7f7c592-7dbd-4543-a2c8-8a350322b442 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:46.025129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e7f7c592-7dbd-4543-a2c8-8a350322b442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:46.202194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1120/2240] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:46.267073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610177cf-c13a-4e8e-8acb-6a159e87baeb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:46.267351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610177cf-c13a-4e8e-8acb-6a159e87baeb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:46.267825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610177cf-c13a-4e8e-8acb-6a159e87baeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:46.292288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-610177cf-c13a-4e8e-8acb-6a159e87baeb None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:46.293593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-610177cf-c13a-4e8e-8acb-6a159e87baeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:46.306950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1121/2241] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:46.309616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7c592-7dbd-4543-a2c8-8a350322b442 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:46.328599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7c592-7dbd-4543-a2c8-8a350322b442 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:46.335771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b0c0d-b4cb-419b-b5a3-02150ae6eddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:46.336479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b0c0d-b4cb-419b-b5a3-02150ae6eddb admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:46.337705 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b0c0d-b4cb-419b-b5a3-02150ae6eddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:46.338136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b0c0d-b4cb-419b-b5a3-02150ae6eddb admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:46.339022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b0c0d-b4cb-419b-b5a3-02150ae6eddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:30:46.339412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b0c0d-b4cb-419b-b5a3-02150ae6eddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:46.341674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b0c0d-b4cb-419b-b5a3-02150ae6eddb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:46.371278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1121/2242] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:30:46.372167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610177cf-c13a-4e8e-8acb-6a159e87baeb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:46.387032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610177cf-c13a-4e8e-8acb-6a159e87baeb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:46.404233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d658e-c5b3-4ecf-8fc8-dc05c897e6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:46.404868 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d658e-c5b3-4ecf-8fc8-dc05c897e6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:46.405780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d658e-c5b3-4ecf-8fc8-dc05c897e6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:46.406304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d658e-c5b3-4ecf-8fc8-dc05c897e6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:46.414942 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d658e-c5b3-4ecf-8fc8-dc05c897e6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:46.424355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d658e-c5b3-4ecf-8fc8-dc05c897e6f1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:46.477542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1122/2243] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:30:46 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:30:46.481635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b0c0d-b4cb-419b-b5a3-02150ae6eddb admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:46.493138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b0c0d-b4cb-419b-b5a3-02150ae6eddb admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:46.574009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:30:46.652328 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa326-cd53-40be-9d96-7270f07052da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:46.653250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa326-cd53-40be-9d96-7270f07052da admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:46.653523 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa326-cd53-40be-9d96-7270f07052da admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:46.653892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa326-cd53-40be-9d96-7270f07052da admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:46.654683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaa326-cd53-40be-9d96-7270f07052da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:46.661610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaa326-cd53-40be-9d96-7270f07052da admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:46.701055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1122/2244] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:30:46 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:30:46.702308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d658e-c5b3-4ecf-8fc8-dc05c897e6f1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:46.715062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d658e-c5b3-4ecf-8fc8-dc05c897e6f1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:46.721921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b3faa-e505-4e20-8f8a-a5fb3c3213d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:46.722908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b3faa-e505-4e20-8f8a-a5fb3c3213d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:46.723399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b3faa-e505-4e20-8f8a-a5fb3c3213d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:46.723900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b3faa-e505-4e20-8f8a-a5fb3c3213d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:46.724759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b3faa-e505-4e20-8f8a-a5fb3c3213d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:46.727353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b3faa-e505-4e20-8f8a-a5fb3c3213d8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:46.749797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1123/2245] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:30:46.753588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa326-cd53-40be-9d96-7270f07052da admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:46.773094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa326-cd53-40be-9d96-7270f07052da admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:46.780988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5964635-54de-4b08-8ff0-b90afc25928d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:46.781616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5964635-54de-4b08-8ff0-b90afc25928d admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:46.783612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5964635-54de-4b08-8ff0-b90afc25928d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:46.783909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5964635-54de-4b08-8ff0-b90afc25928d admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:46.786209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5964635-54de-4b08-8ff0-b90afc25928d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:46.792128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5964635-54de-4b08-8ff0-b90afc25928d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:46.820169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1123/2246] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:30:46.821227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b3faa-e505-4e20-8f8a-a5fb3c3213d8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:46.838913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b3faa-e505-4e20-8f8a-a5fb3c3213d8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:46.847094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a99d7-a4aa-4eec-9141-ed0a8c3590fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:46.847947 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a99d7-a4aa-4eec-9141-ed0a8c3590fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:46.848413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a99d7-a4aa-4eec-9141-ed0a8c3590fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:46.848874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a99d7-a4aa-4eec-9141-ed0a8c3590fd admin admin] PATH_INFO: `/v3/projects/9d54c170376c40b2aa0593345c6b3782/users/261a29da49714b67a5275c26cb8879ea/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:46.886045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a99d7-a4aa-4eec-9141-ed0a8c3590fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d54c170376c40b2aa0593345c6b3782, user_id=261a29da49714b67a5275c26cb8879ea, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:46.889486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a99d7-a4aa-4eec-9141-ed0a8c3590fd admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:46.896818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:30:46.919766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1124/2247] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:30:46 2022] PUT /identity/v3/projects/9d54c170376c40b2aa0593345c6b3782/users/261a29da49714b67a5275c26cb8879ea/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:30:46.930656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e8810-1102-49ef-9d3d-1da98f5e610a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:46.930999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e8810-1102-49ef-9d3d-1da98f5e610a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:46.931279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e8810-1102-49ef-9d3d-1da98f5e610a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:46.988892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ea9e8810-1102-49ef-9d3d-1da98f5e610a None None] MFA Rules not processed for user `261a29da49714b67a5275c26cb8879ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:46.991736 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ea9e8810-1102-49ef-9d3d-1da98f5e610a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:47.107176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1124/2248] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:30:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:47.114089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a99d7-a4aa-4eec-9141-ed0a8c3590fd admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:47.115888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6e0a99d7-a4aa-4eec-9141-ed0a8c3590fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:47.226481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a99d7-a4aa-4eec-9141-ed0a8c3590fd admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:47.231245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73afdab-a807-4dcb-9713-5abd0b5687d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:47.232554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73afdab-a807-4dcb-9713-5abd0b5687d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:47.234688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73afdab-a807-4dcb-9713-5abd0b5687d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:47.234688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73afdab-a807-4dcb-9713-5abd0b5687d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:47.246551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73afdab-a807-4dcb-9713-5abd0b5687d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:47.248627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73afdab-a807-4dcb-9713-5abd0b5687d5 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:47.262049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1125/2249] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:30:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:30:48.762927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f46b1-a5bb-45b5-989a-d5c1e6325b8d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:48.763417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f46b1-a5bb-45b5-989a-d5c1e6325b8d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:48.763613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f46b1-a5bb-45b5-989a-d5c1e6325b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:48.782779 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c95f46b1-a5bb-45b5-989a-d5c1e6325b8d None None] MFA Rules not processed for user `261a29da49714b67a5275c26cb8879ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:48.783970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c95f46b1-a5bb-45b5-989a-d5c1e6325b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:48.791136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1125/2250] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:30:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:57.256649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:30:58.583396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c4a94-b67d-4fbd-9db7-440e277c9ece None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:58.584619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c4a94-b67d-4fbd-9db7-440e277c9ece None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:58.585551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c4a94-b67d-4fbd-9db7-440e277c9ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:58.607827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-018c4a94-b67d-4fbd-9db7-440e277c9ece None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:58.608894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-018c4a94-b67d-4fbd-9db7-440e277c9ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:58.687693 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1126/2251] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:30:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:30:58.702089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f46b1-a5bb-45b5-989a-d5c1e6325b8d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:30:58.729977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27ee53-bae3-457a-85f8-657d30360092 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:58.730288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c95f46b1-a5bb-45b5-989a-d5c1e6325b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:58.730427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27ee53-bae3-457a-85f8-657d30360092 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:58.730427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27ee53-bae3-457a-85f8-657d30360092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:58.748105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7e27ee53-bae3-457a-85f8-657d30360092 None None] MFA Rules not processed for user `07df6c0689084d3d95338d3fe0627b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:30:58.749619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7e27ee53-bae3-457a-85f8-657d30360092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:30:58.796537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f46b1-a5bb-45b5-989a-d5c1e6325b8d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:30:58.801566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81938f-d5f9-4c66-815e-06d2a151cf12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:30:58.802233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81938f-d5f9-4c66-815e-06d2a151cf12 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:30:58.802606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81938f-d5f9-4c66-815e-06d2a151cf12 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:30:58.802904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81938f-d5f9-4c66-815e-06d2a151cf12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:30:58.809972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1127/2252] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:30:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:30:58.813125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81938f-d5f9-4c66-815e-06d2a151cf12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:30:58.814745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81938f-d5f9-4c66-815e-06d2a151cf12 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:30:58.824273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1126/2253] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:30:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:31:08.802054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=50 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:08.821134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:16.490354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cd22b-a4d1-45bd-81ff-72ffd5857a51 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:16.491471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cd22b-a4d1-45bd-81ff-72ffd5857a51 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:16.491652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cd22b-a4d1-45bd-81ff-72ffd5857a51 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:16.493085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1128/2254] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:31:16.499413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357f13a-d6d1-46c6-9624-b4ec24e119e0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:16.500202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357f13a-d6d1-46c6-9624-b4ec24e119e0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:16.500517 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357f13a-d6d1-46c6-9624-b4ec24e119e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:16.520717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a357f13a-d6d1-46c6-9624-b4ec24e119e0 None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:16.521992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a357f13a-d6d1-46c6-9624-b4ec24e119e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:16.601785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1127/2255] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:31:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:16.614236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cd22b-a4d1-45bd-81ff-72ffd5857a51 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:16.617361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-536cd22b-a4d1-45bd-81ff-72ffd5857a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:16.686310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cd22b-a4d1-45bd-81ff-72ffd5857a51 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:16.689800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00297c2f-c7d6-4f9b-a899-0d15967bc6b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5305f8a5d2b540de91d3caedad454bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:16.690431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00297c2f-c7d6-4f9b-a899-0d15967bc6b0 service glance] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:16.690706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00297c2f-c7d6-4f9b-a899-0d15967bc6b0 service glance] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:16.690974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00297c2f-c7d6-4f9b-a899-0d15967bc6b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:16.702643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00297c2f-c7d6-4f9b-a899-0d15967bc6b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:16.704081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00297c2f-c7d6-4f9b-a899-0d15967bc6b0 service glance] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:16.718252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1129/2256] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:31:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:31:16.790200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f85d9-c2a6-47bf-a205-4f7ee228c16c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:16.790488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f85d9-c2a6-47bf-a205-4f7ee228c16c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:16.790761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f85d9-c2a6-47bf-a205-4f7ee228c16c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:16.792248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1128/2257] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 05:31:16.799586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fcf18-1a94-4263-a95c-6677791f83a5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:16.800033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fcf18-1a94-4263-a95c-6677791f83a5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:16.800322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fcf18-1a94-4263-a95c-6677791f83a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:16.820867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-289fcf18-1a94-4263-a95c-6677791f83a5 None None] MFA Rules not processed for user `c07403facfaf4c71ab5324af6c44f52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:16.822084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-289fcf18-1a94-4263-a95c-6677791f83a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:16.847942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ee436-795f-4710-9f7e-5226a09ca156 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:16.848331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ee436-795f-4710-9f7e-5226a09ca156 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:16.849047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ee436-795f-4710-9f7e-5226a09ca156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:16.869253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-874ee436-795f-4710-9f7e-5226a09ca156 None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:16.870342 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-874ee436-795f-4710-9f7e-5226a09ca156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:16.876281 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1130/2258] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:31:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:16.877419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1129/2259] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:31:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:31:16.887936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fcf18-1a94-4263-a95c-6677791f83a5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:16.889292 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-289fcf18-1a94-4263-a95c-6677791f83a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:16.890182 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ee436-795f-4710-9f7e-5226a09ca156 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:16.900400 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fcf18-1a94-4263-a95c-6677791f83a5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:16.902715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ee436-795f-4710-9f7e-5226a09ca156 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:16.904163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5e790-0c3d-460d-8056-ffc1a4dc9aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:16.904762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5e790-0c3d-460d-8056-ffc1a4dc9aee service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:16.905042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5e790-0c3d-460d-8056-ffc1a4dc9aee service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:16.905314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5e790-0c3d-460d-8056-ffc1a4dc9aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:16.907621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbba09-d682-4d34-a3a4-605b06acf849 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:16.908211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbba09-d682-4d34-a3a4-605b06acf849 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:16.908489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbba09-d682-4d34-a3a4-605b06acf849 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:16.908760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbba09-d682-4d34-a3a4-605b06acf849 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:16.912406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5e790-0c3d-460d-8056-ffc1a4dc9aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:16.914388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5e790-0c3d-460d-8056-ffc1a4dc9aee service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:16.918303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bbba09-d682-4d34-a3a4-605b06acf849 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:16.919845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bbba09-d682-4d34-a3a4-605b06acf849 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:16.922168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1131/2260] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:31:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:31:16.932126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1130/2261] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:31:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 05:31:17.014990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4668c-65f5-47d2-acfd-ce038f83b110 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:17.015388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4668c-65f5-47d2-acfd-ce038f83b110 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:17.015763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4668c-65f5-47d2-acfd-ce038f83b110 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:17.017248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1132/2262] 10.222.0.18 () {54 vars in 921 bytes} [Wed Sep 21 05:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 05:31:17.023454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63ffc0-c2a0-46ae-81dd-1ba8575ee9e9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:17.023814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63ffc0-c2a0-46ae-81dd-1ba8575ee9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:17.024160 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63ffc0-c2a0-46ae-81dd-1ba8575ee9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:17.046685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7b63ffc0-c2a0-46ae-81dd-1ba8575ee9e9 None None] MFA Rules not processed for user `d9168d8bbf8b43299a7748c9faa099b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:17.047894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7b63ffc0-c2a0-46ae-81dd-1ba8575ee9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:17.058344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1131/2263] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Sep 21 05:31:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:17.069554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4668c-65f5-47d2-acfd-ce038f83b110 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:17.072136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c6a4668c-65f5-47d2-acfd-ce038f83b110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:17.086425 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4668c-65f5-47d2-acfd-ce038f83b110 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:17.090169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aa0af-31ab-4319-b3c6-7b8e609eff78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:17.090768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aa0af-31ab-4319-b3c6-7b8e609eff78 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:17.091721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aa0af-31ab-4319-b3c6-7b8e609eff78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:17.091995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aa0af-31ab-4319-b3c6-7b8e609eff78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:17.103158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aa0af-31ab-4319-b3c6-7b8e609eff78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:17.104851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aa0af-31ab-4319-b3c6-7b8e609eff78 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:17.116267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1133/2264] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:31:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:31:17.515609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a2b93-5180-4691-9eb0-5bdd1538180f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:17.516117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a2b93-5180-4691-9eb0-5bdd1538180f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:17.516244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a2b93-5180-4691-9eb0-5bdd1538180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:17.535415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f18a2b93-5180-4691-9eb0-5bdd1538180f None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:17.536620 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f18a2b93-5180-4691-9eb0-5bdd1538180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:17.544457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1132/2265] 10.222.0.18 () {58 vars in 967 bytes} [Wed Sep 21 05:31:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:17.558102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aa0af-31ab-4319-b3c6-7b8e609eff78 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:17.569630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aa0af-31ab-4319-b3c6-7b8e609eff78 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:17.574863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195426d-c98e-4c95-863e-7a5dcdde3cdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:17.575487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195426d-c98e-4c95-863e-7a5dcdde3cdf service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:17.575765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195426d-c98e-4c95-863e-7a5dcdde3cdf service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:17.576034 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195426d-c98e-4c95-863e-7a5dcdde3cdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:17.584711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195426d-c98e-4c95-863e-7a5dcdde3cdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:17.586331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195426d-c98e-4c95-863e-7a5dcdde3cdf service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:17.594914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1134/2266] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:31:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:31:17.848401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fddd0-eb25-44d9-a949-a3236c42631d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:17.848688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fddd0-eb25-44d9-a949-a3236c42631d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:17.848966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fddd0-eb25-44d9-a949-a3236c42631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:17.868976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-876fddd0-eb25-44d9-a949-a3236c42631d None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:17.870201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-876fddd0-eb25-44d9-a949-a3236c42631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:17.877139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1133/2267] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:31:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:17.889076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195426d-c98e-4c95-863e-7a5dcdde3cdf service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:17.899414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195426d-c98e-4c95-863e-7a5dcdde3cdf service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:17.904510 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d6801-d8d3-40a2-90ef-2e87b8ba7025 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:17.905074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d6801-d8d3-40a2-90ef-2e87b8ba7025 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:17.905355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d6801-d8d3-40a2-90ef-2e87b8ba7025 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:17.905630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d6801-d8d3-40a2-90ef-2e87b8ba7025 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:17.914185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d6801-d8d3-40a2-90ef-2e87b8ba7025 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:17.915761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d6801-d8d3-40a2-90ef-2e87b8ba7025 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:17.923713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1135/2268] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:31:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:31:18.236503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc6bc3-3808-491c-a020-4042f1ebdb9d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:18.236794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc6bc3-3808-491c-a020-4042f1ebdb9d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:18.237063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc6bc3-3808-491c-a020-4042f1ebdb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:18.255609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8fdc6bc3-3808-491c-a020-4042f1ebdb9d None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:18.256790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8fdc6bc3-3808-491c-a020-4042f1ebdb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:18.263749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1134/2269] 10.222.0.18 () {58 vars in 967 bytes} [Wed Sep 21 05:31:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:18.276054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d6801-d8d3-40a2-90ef-2e87b8ba7025 service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:18.286480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d6801-d8d3-40a2-90ef-2e87b8ba7025 service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:18.292099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de45feb-45ba-4e18-8313-1bb6a34e82bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:18.292686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de45feb-45ba-4e18-8313-1bb6a34e82bc service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:18.292967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de45feb-45ba-4e18-8313-1bb6a34e82bc service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:18.293240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de45feb-45ba-4e18-8313-1bb6a34e82bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:18.301061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de45feb-45ba-4e18-8313-1bb6a34e82bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:18.302714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de45feb-45ba-4e18-8313-1bb6a34e82bc service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:18.310503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1136/2270] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:31:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:31:20.541121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc6bc3-3808-491c-a020-4042f1ebdb9d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:20.604406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc6bc3-3808-491c-a020-4042f1ebdb9d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:20.608785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:20.609408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:20.609685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:20.609980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] PATH_INFO: `/v3/users/261a29da49714b67a5275c26cb8879ea` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:20.611089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=261a29da49714b67a5275c26cb8879ea)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:20.613287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:20.658851 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '261a29da49714b67a5275c26cb8879ea'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:20.660476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:20.667499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '261a29da49714b67a5275c26cb8879ea'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:20.668506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '261a29da49714b67a5275c26cb8879ea'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:20.669104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '261a29da49714b67a5275c26cb8879ea'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:20.676051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '261a29da49714b67a5275c26cb8879ea'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:20.694050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1135/2271] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:31:20 2022] DELETE /identity/v3/users/261a29da49714b67a5275c26cb8879ea => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:31:20.698663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e2c03-3657-47c7-9741-5114cc3cf4fc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:20.699112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e2c03-3657-47c7-9741-5114cc3cf4fc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:20.699204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e2c03-3657-47c7-9741-5114cc3cf4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:20.718158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-651e2c03-3657-47c7-9741-5114cc3cf4fc None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:20.719355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-651e2c03-3657-47c7-9741-5114cc3cf4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:20.759373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:20.777689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1137/2272] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:20.787132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:20.789062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:20.845148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516e69b-b721-421f-a163-e76ae1e32c7c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:20.848807 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4210ae6-b055-4a74-b11b-15af60393e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:20.849402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4210ae6-b055-4a74-b11b-15af60393e07 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:20.849676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4210ae6-b055-4a74-b11b-15af60393e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:20.849958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4210ae6-b055-4a74-b11b-15af60393e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:20.858897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4210ae6-b055-4a74-b11b-15af60393e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:20.860303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4210ae6-b055-4a74-b11b-15af60393e07 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:20.870276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1136/2273] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:31:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:31:21.041813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e2c03-3657-47c7-9741-5114cc3cf4fc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:21.043483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-651e2c03-3657-47c7-9741-5114cc3cf4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:21.055912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e2c03-3657-47c7-9741-5114cc3cf4fc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:21.060593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:21.061299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.061651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.062430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] PATH_INFO: `/v3/projects/9d54c170376c40b2aa0593345c6b3782` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.063254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d54c170376c40b2aa0593345c6b3782)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:21.065199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:21.111923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d54c170376c40b2aa0593345c6b3782', 'request_id': 'req-f4be8795-29ba-4827-b685-1bc66e4ccee5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:21.117536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d54c170376c40b2aa0593345c6b3782', 'request_id': 'req-f4be8795-29ba-4827-b685-1bc66e4ccee5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:21.124729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d54c170376c40b2aa0593345c6b3782', 'request_id': 'req-f4be8795-29ba-4827-b685-1bc66e4ccee5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:21.136402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] The token cache is being invalidate because project 9d54c170376c40b2aa0593345c6b3782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:31:21.136693 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:21.137307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4be8795-29ba-4827-b685-1bc66e4ccee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:31:21.138786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1138/2274] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:31:21 2022] DELETE /identity/v3/projects/9d54c170376c40b2aa0593345c6b3782 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:31:21.143787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7328332-74a6-4f2b-b6e3-224741319482 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.144092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7328332-74a6-4f2b-b6e3-224741319482 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.144347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7328332-74a6-4f2b-b6e3-224741319482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.163858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b7328332-74a6-4f2b-b6e3-224741319482 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:21.165043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b7328332-74a6-4f2b-b6e3-224741319482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:21.223937 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1137/2275] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:21.231556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e3e89-eb01-410e-b106-c5045d7a54ef None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.231839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e3e89-eb01-410e-b106-c5045d7a54ef None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.232112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e3e89-eb01-410e-b106-c5045d7a54ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.252256 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c12e3e89-eb01-410e-b106-c5045d7a54ef None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:21.253457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c12e3e89-eb01-410e-b106-c5045d7a54ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:21.262494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1139/2276] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:21.266813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7328332-74a6-4f2b-b6e3-224741319482 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:21.276764 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7328332-74a6-4f2b-b6e3-224741319482 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:21.281382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e38037-9096-451c-8994-6a1c3be0abc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:21.282003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e38037-9096-451c-8994-6a1c3be0abc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.282286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e38037-9096-451c-8994-6a1c3be0abc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.282556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e38037-9096-451c-8994-6a1c3be0abc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.283221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e38037-9096-451c-8994-6a1c3be0abc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:31:21.283522 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e38037-9096-451c-8994-6a1c3be0abc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:21.285163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e38037-9096-451c-8994-6a1c3be0abc5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:21.300140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1138/2277] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:31:21.301497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e3e89-eb01-410e-b106-c5045d7a54ef None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:21.311374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e3e89-eb01-410e-b106-c5045d7a54ef None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:21.316024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a344936-48cd-422e-8d3e-b42b66530d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:21.316621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a344936-48cd-422e-8d3e-b42b66530d97 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.316898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a344936-48cd-422e-8d3e-b42b66530d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.317165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a344936-48cd-422e-8d3e-b42b66530d97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.317824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a344936-48cd-422e-8d3e-b42b66530d97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:21.320303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a344936-48cd-422e-8d3e-b42b66530d97 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:21.348182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1140/2278] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:31:21 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:31:21.353443 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e38037-9096-451c-8994-6a1c3be0abc5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:21.363837 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e38037-9096-451c-8994-6a1c3be0abc5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:21.412385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:21.419854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bbcf7-632a-404e-aa33-0ed7cd539a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:21.420571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bbcf7-632a-404e-aa33-0ed7cd539a90 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.421052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bbcf7-632a-404e-aa33-0ed7cd539a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.421493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bbcf7-632a-404e-aa33-0ed7cd539a90 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.422230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bbcf7-632a-404e-aa33-0ed7cd539a90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:21.424312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bbcf7-632a-404e-aa33-0ed7cd539a90 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:21.446020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1139/2279] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:31:21 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:31:21.450235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a344936-48cd-422e-8d3e-b42b66530d97 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:21.460601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a344936-48cd-422e-8d3e-b42b66530d97 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:21.465117 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c1c8d-1d3e-4864-8b16-36ad3a41e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:21.465705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c1c8d-1d3e-4864-8b16-36ad3a41e412 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.466002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c1c8d-1d3e-4864-8b16-36ad3a41e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.466272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c1c8d-1d3e-4864-8b16-36ad3a41e412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.466902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c1c8d-1d3e-4864-8b16-36ad3a41e412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:21.468480 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c1c8d-1d3e-4864-8b16-36ad3a41e412 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:21.478186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1141/2280] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:31:21.482091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bbcf7-632a-404e-aa33-0ed7cd539a90 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:21.492478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bbcf7-632a-404e-aa33-0ed7cd539a90 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:21.497060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3d6d3-2aba-4bfa-b57d-b525d876fee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:21.497647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3d6d3-2aba-4bfa-b57d-b525d876fee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.497924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3d6d3-2aba-4bfa-b57d-b525d876fee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.498223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3d6d3-2aba-4bfa-b57d-b525d876fee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.498850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3d6d3-2aba-4bfa-b57d-b525d876fee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:21.500402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3d6d3-2aba-4bfa-b57d-b525d876fee2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:21.511241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1140/2281] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:31:21.515338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c1c8d-1d3e-4864-8b16-36ad3a41e412 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:21.525113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c1c8d-1d3e-4864-8b16-36ad3a41e412 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:21.529707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc866977-6c4e-4060-83c8-2f0dd9175683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:21.530387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc866977-6c4e-4060-83c8-2f0dd9175683 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.530665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc866977-6c4e-4060-83c8-2f0dd9175683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.530939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc866977-6c4e-4060-83c8-2f0dd9175683 admin admin] PATH_INFO: `/v3/projects/8d9fd11e99ac438e8308d62d7637b007/users/9d05e7bf6ff04193849987316fcedb81/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.545048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc866977-6c4e-4060-83c8-2f0dd9175683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9fd11e99ac438e8308d62d7637b007, user_id=9d05e7bf6ff04193849987316fcedb81, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:21.547689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc866977-6c4e-4060-83c8-2f0dd9175683 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:21.560785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1142/2282] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:31:21 2022] PUT /identity/v3/projects/8d9fd11e99ac438e8308d62d7637b007/users/9d05e7bf6ff04193849987316fcedb81/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:31:21.565646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3f238-cb90-403f-8783-feebfef62df8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.566142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3f238-cb90-403f-8783-feebfef62df8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.566239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3f238-cb90-403f-8783-feebfef62df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.587003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-32a3f238-cb90-403f-8783-feebfef62df8 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:21.588204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-32a3f238-cb90-403f-8783-feebfef62df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:21.650288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1141/2283] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:31:21.660415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc866977-6c4e-4060-83c8-2f0dd9175683 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:21.662769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fc866977-6c4e-4060-83c8-2f0dd9175683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:21.719173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc866977-6c4e-4060-83c8-2f0dd9175683 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:21.722871 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662c237-1551-4c61-aa27-76dcd29bd3d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:21.723463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662c237-1551-4c61-aa27-76dcd29bd3d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:21.723738 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662c237-1551-4c61-aa27-76dcd29bd3d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:21.724007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662c237-1551-4c61-aa27-76dcd29bd3d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:21.733171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662c237-1551-4c61-aa27-76dcd29bd3d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:21.734624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662c237-1551-4c61-aa27-76dcd29bd3d3 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:21.744576 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1143/2284] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:31:25.854125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f725e-1902-4d8a-8b9d-f8f42a21f247 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:25.854565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f725e-1902-4d8a-8b9d-f8f42a21f247 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:25.854832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f725e-1902-4d8a-8b9d-f8f42a21f247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:25.886255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-529f725e-1902-4d8a-8b9d-f8f42a21f247 None None] MFA Rules not processed for user `9d05e7bf6ff04193849987316fcedb81`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:25.887555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-529f725e-1902-4d8a-8b9d-f8f42a21f247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:25.973876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1142/2285] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:31:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:25.978646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662c237-1551-4c61-aa27-76dcd29bd3d3 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:25.988640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662c237-1551-4c61-aa27-76dcd29bd3d3 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:25.993790 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f283b-97fa-454a-9b0e-6e180ffb8b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:25.994444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f283b-97fa-454a-9b0e-6e180ffb8b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:25.994724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f283b-97fa-454a-9b0e-6e180ffb8b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:25.994994 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f283b-97fa-454a-9b0e-6e180ffb8b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:25.995620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f283b-97fa-454a-9b0e-6e180ffb8b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:25.997525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f283b-97fa-454a-9b0e-6e180ffb8b63 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:26.003420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:26.031106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1144/2286] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:31:25 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:31:26.035008 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f725e-1902-4d8a-8b9d-f8f42a21f247 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:26.046499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f725e-1902-4d8a-8b9d-f8f42a21f247 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:26.106119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070dcb5-2ca8-40ca-ba64-9fdcd64fb191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:26.106667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070dcb5-2ca8-40ca-ba64-9fdcd64fb191 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:26.106949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070dcb5-2ca8-40ca-ba64-9fdcd64fb191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:26.107229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070dcb5-2ca8-40ca-ba64-9fdcd64fb191 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:26.107875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070dcb5-2ca8-40ca-ba64-9fdcd64fb191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:26.109791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070dcb5-2ca8-40ca-ba64-9fdcd64fb191 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:26.121244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:26.135954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1143/2287] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:31:26 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:31:26.140015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f283b-97fa-454a-9b0e-6e180ffb8b63 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:26.150942 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f283b-97fa-454a-9b0e-6e180ffb8b63 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:26.155583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70555010-04f9-4934-95a4-37634d5e5dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:26.156193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70555010-04f9-4934-95a4-37634d5e5dad admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:26.156472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70555010-04f9-4934-95a4-37634d5e5dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:26.156746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70555010-04f9-4934-95a4-37634d5e5dad admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:26.157424 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70555010-04f9-4934-95a4-37634d5e5dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:26.159125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70555010-04f9-4934-95a4-37634d5e5dad admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:26.170716 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1145/2288] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:31:26.174441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070dcb5-2ca8-40ca-ba64-9fdcd64fb191 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:26.185579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070dcb5-2ca8-40ca-ba64-9fdcd64fb191 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:26.190988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39ae6b-8b45-4375-ad6c-bf6fb64dc483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:26.191829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39ae6b-8b45-4375-ad6c-bf6fb64dc483 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:26.192179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39ae6b-8b45-4375-ad6c-bf6fb64dc483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:26.192524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39ae6b-8b45-4375-ad6c-bf6fb64dc483 admin admin] PATH_INFO: `/v3/projects/d554d459a8904812addae8aad925b8a2/users/dfb7d4564524432e8b54ae65f350f3f9/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:26.208529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39ae6b-8b45-4375-ad6c-bf6fb64dc483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d554d459a8904812addae8aad925b8a2, user_id=dfb7d4564524432e8b54ae65f350f3f9, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:26.211585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39ae6b-8b45-4375-ad6c-bf6fb64dc483 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:26.225918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1144/2289] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:31:26 2022] PUT /identity/v3/projects/d554d459a8904812addae8aad925b8a2/users/dfb7d4564524432e8b54ae65f350f3f9/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:31:26.229860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70555010-04f9-4934-95a4-37634d5e5dad admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:26.240061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70555010-04f9-4934-95a4-37634d5e5dad admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:26.314210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902f3c8-494a-442b-aac5-32ee4a27a632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:26.315011 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902f3c8-494a-442b-aac5-32ee4a27a632 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:26.315130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902f3c8-494a-442b-aac5-32ee4a27a632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:26.315382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902f3c8-494a-442b-aac5-32ee4a27a632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:26.317807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902f3c8-494a-442b-aac5-32ee4a27a632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:26.318155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902f3c8-494a-442b-aac5-32ee4a27a632 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:26.333547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1146/2290] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:31:26.337624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39ae6b-8b45-4375-ad6c-bf6fb64dc483 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:26.356535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39ae6b-8b45-4375-ad6c-bf6fb64dc483 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:26.375752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b7354-7854-47ea-bf2f-51d4b9b5c86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:26.376592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b7354-7854-47ea-bf2f-51d4b9b5c86a admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:26.376882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b7354-7854-47ea-bf2f-51d4b9b5c86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:26.377165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b7354-7854-47ea-bf2f-51d4b9b5c86a admin admin] PATH_INFO: `/v3/projects/d554d459a8904812addae8aad925b8a2/users/dfb7d4564524432e8b54ae65f350f3f9/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:26.390215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b7354-7854-47ea-bf2f-51d4b9b5c86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d554d459a8904812addae8aad925b8a2, user_id=dfb7d4564524432e8b54ae65f350f3f9, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:26.393277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b7354-7854-47ea-bf2f-51d4b9b5c86a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:26.410605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1145/2291] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:31:26 2022] PUT /identity/v3/projects/d554d459a8904812addae8aad925b8a2/users/dfb7d4564524432e8b54ae65f350f3f9/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:31:31.252095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7196cda-37b1-4afd-934b-309cc406dba9 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:31.252610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7196cda-37b1-4afd-934b-309cc406dba9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:31.252692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7196cda-37b1-4afd-934b-309cc406dba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:31.284495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e7196cda-37b1-4afd-934b-309cc406dba9 None None] MFA Rules not processed for user `dfb7d4564524432e8b54ae65f350f3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:31.285723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e7196cda-37b1-4afd-934b-309cc406dba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:31.437432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1147/2292] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:31:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:31.438258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b7354-7854-47ea-bf2f-51d4b9b5c86a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:31.441018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9e4b7354-7854-47ea-bf2f-51d4b9b5c86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:31.538916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b7354-7854-47ea-bf2f-51d4b9b5c86a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:31.547660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74ef6d-b50b-4d1b-bfea-117a12a2f372 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:31.548330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74ef6d-b50b-4d1b-bfea-117a12a2f372 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:31.548619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74ef6d-b50b-4d1b-bfea-117a12a2f372 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:31.548896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74ef6d-b50b-4d1b-bfea-117a12a2f372 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:31.618634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74ef6d-b50b-4d1b-bfea-117a12a2f372 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:31.620423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74ef6d-b50b-4d1b-bfea-117a12a2f372 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:31.631361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1146/2293] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:31:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:31:32.603412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462d79c-3497-475e-b6cf-93e7c079a3db None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:32.603862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462d79c-3497-475e-b6cf-93e7c079a3db None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:32.604595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462d79c-3497-475e-b6cf-93e7c079a3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:32.644329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d462d79c-3497-475e-b6cf-93e7c079a3db None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:32.645755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d462d79c-3497-475e-b6cf-93e7c079a3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:32.724046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1148/2294] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:31:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:31:32.750483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74ef6d-b50b-4d1b-bfea-117a12a2f372 service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:31:32.752104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4c74ef6d-b50b-4d1b-bfea-117a12a2f372 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:32.817525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:32.846866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74ef6d-b50b-4d1b-bfea-117a12a2f372 service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:31:32.856186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a4e02-e6d2-4f1f-99a6-ca850ce007c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:31:32.856848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a4e02-e6d2-4f1f-99a6-ca850ce007c3 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:32.857129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a4e02-e6d2-4f1f-99a6-ca850ce007c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:32.857431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a4e02-e6d2-4f1f-99a6-ca850ce007c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:32.880375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a4e02-e6d2-4f1f-99a6-ca850ce007c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:31:32.882272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a4e02-e6d2-4f1f-99a6-ca850ce007c3 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:31:32.903166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1147/2295] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:31:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:31:42.714343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:42.889026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=8 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:31:53.265322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d1307-83f1-4862-a0f2-4f48890b4d6f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:31:53.265869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d1307-83f1-4862-a0f2-4f48890b4d6f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:31:53.265869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d1307-83f1-4862-a0f2-4f48890b4d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:31:53.284162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5b2d1307-83f1-4862-a0f2-4f48890b4d6f None None] MFA Rules not processed for user `9d05e7bf6ff04193849987316fcedb81`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:31:53.285342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5b2d1307-83f1-4862-a0f2-4f48890b4d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:31:53.292545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1149/2296] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:31:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:32:02.239102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f987962-f79e-46d4-af13-6adead57ed79 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:02.240077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f987962-f79e-46d4-af13-6adead57ed79 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:02.240569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f987962-f79e-46d4-af13-6adead57ed79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:02.276900 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1f987962-f79e-46d4-af13-6adead57ed79 None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:32:02.278477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1f987962-f79e-46d4-af13-6adead57ed79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:02.299229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1148/2297] 10.222.0.18 () {58 vars in 967 bytes} [Wed Sep 21 05:32:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:32:02.300959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d1307-83f1-4862-a0f2-4f48890b4d6f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:02.319219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d1307-83f1-4862-a0f2-4f48890b4d6f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:02.325690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4fed0-be34-46fd-84f8-4d272d43f6a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:02.326759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4fed0-be34-46fd-84f8-4d272d43f6a3 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:02.327193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4fed0-be34-46fd-84f8-4d272d43f6a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:02.329590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4fed0-be34-46fd-84f8-4d272d43f6a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:02.402907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4fed0-be34-46fd-84f8-4d272d43f6a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:02.413528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4fed0-be34-46fd-84f8-4d272d43f6a3 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:02.461349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1150/2298] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:32:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:32:10.671786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f987962-f79e-46d4-af13-6adead57ed79 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:10.674355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1f987962-f79e-46d4-af13-6adead57ed79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:10.728217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f987962-f79e-46d4-af13-6adead57ed79 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:10.731793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dc718-1b18-4068-94c5-ab6d2a1c8e69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:10.732398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dc718-1b18-4068-94c5-ab6d2a1c8e69 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:10.732676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dc718-1b18-4068-94c5-ab6d2a1c8e69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:10.732953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dc718-1b18-4068-94c5-ab6d2a1c8e69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:10.734770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-580dc718-1b18-4068-94c5-ab6d2a1c8e69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:10.748090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580dc718-1b18-4068-94c5-ab6d2a1c8e69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:10.749488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580dc718-1b18-4068-94c5-ab6d2a1c8e69 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:10.764895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1149/2299] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:32:11.895224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4fed0-be34-46fd-84f8-4d272d43f6a3 service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:11.905209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4fed0-be34-46fd-84f8-4d272d43f6a3 service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:11.908831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:11.909426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:11.909700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:11.910003 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:11.911714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:11.924661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:11.926135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:11.936635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1151/2300] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:32:18.326777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dc718-1b18-4068-94c5-ab6d2a1c8e69 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:18.336819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dc718-1b18-4068-94c5-ab6d2a1c8e69 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:18.340736 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f674140-62c8-4fbe-8981-d199a2dc3058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:18.341361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f674140-62c8-4fbe-8981-d199a2dc3058 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:18.341641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f674140-62c8-4fbe-8981-d199a2dc3058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:18.341915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f674140-62c8-4fbe-8981-d199a2dc3058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:18.351052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f674140-62c8-4fbe-8981-d199a2dc3058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:18.352467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f674140-62c8-4fbe-8981-d199a2dc3058 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:18.362067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1150/2301] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:32:19.531533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:19.533027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:19.613297 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5235bf-e329-4393-9bef-d3660079d802 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:19.619661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc195f-c17d-45a3-afd9-a0ada0ade810 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:19.620504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc195f-c17d-45a3-afd9-a0ada0ade810 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:19.620989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc195f-c17d-45a3-afd9-a0ada0ade810 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:19.621602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc195f-c17d-45a3-afd9-a0ada0ade810 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:19.624625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f674140-62c8-4fbe-8981-d199a2dc3058 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:19.626635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0f674140-62c8-4fbe-8981-d199a2dc3058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:19.630143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f2dc195f-c17d-45a3-afd9-a0ada0ade810 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:19.643751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc195f-c17d-45a3-afd9-a0ada0ade810 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:19.645680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc195f-c17d-45a3-afd9-a0ada0ade810 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:19.657581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1152/2302] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:32:19.708908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f674140-62c8-4fbe-8981-d199a2dc3058 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:19.714598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:19.715351 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:19.715642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:19.715924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] PATH_INFO: `/v3/users/07df6c0689084d3d95338d3fe0627b1b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:19.717068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07df6c0689084d3d95338d3fe0627b1b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:19.719252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:19.768695 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07df6c0689084d3d95338d3fe0627b1b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:19.776762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07df6c0689084d3d95338d3fe0627b1b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:19.777406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07df6c0689084d3d95338d3fe0627b1b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:19.778374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07df6c0689084d3d95338d3fe0627b1b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:19.785598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e4d2e16-2aea-469f-9906-6fadf28d8b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07df6c0689084d3d95338d3fe0627b1b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:19.805064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1151/2303] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:32:19 2022] DELETE /identity/v3/users/07df6c0689084d3d95338d3fe0627b1b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:32:19.987965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc195f-c17d-45a3-afd9-a0ada0ade810 service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:20.000139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc195f-c17d-45a3-afd9-a0ada0ade810 service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:20.067497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:20.068064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:20.068321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:20.068588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] PATH_INFO: `/v3/projects/6deb1edc63ed425eafbea3dc1dded3ac` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:20.070262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6deb1edc63ed425eafbea3dc1dded3ac)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:20.072179 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:20.120709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6deb1edc63ed425eafbea3dc1dded3ac', 'request_id': 'req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:20.126369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6deb1edc63ed425eafbea3dc1dded3ac', 'request_id': 'req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:20.128167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:32:20.136321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6deb1edc63ed425eafbea3dc1dded3ac', 'request_id': 'req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:20.153192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] The token cache is being invalidate because project 6deb1edc63ed425eafbea3dc1dded3ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:32:20.153515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:20.154280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:32:20.155976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1153/2304] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:32:19 2022] DELETE /identity/v3/projects/6deb1edc63ed425eafbea3dc1dded3ac => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:32:22.112699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad31d03-e3bb-4065-b91f-198c9d9d1233 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:22.113265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad31d03-e3bb-4065-b91f-198c9d9d1233 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:22.113388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad31d03-e3bb-4065-b91f-198c9d9d1233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:22.122735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:32:22.138863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fad31d03-e3bb-4065-b91f-198c9d9d1233 None None] MFA Rules not processed for user `1ec24729b04947b7b05cc6dcd787eb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:32:22.140080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fad31d03-e3bb-4065-b91f-198c9d9d1233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:22.195594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1152/2305] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:32:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:32:27.405783 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:27.407189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:27.482122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2bf93-3040-45ae-9f0d-70a1154ee10b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:27.487922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae065d-5c9f-423b-9840-608f45476b6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:27.488528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae065d-5c9f-423b-9840-608f45476b6b service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:27.488809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae065d-5c9f-423b-9840-608f45476b6b service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:27.489082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae065d-5c9f-423b-9840-608f45476b6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:27.490808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5cae065d-5c9f-423b-9840-608f45476b6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:27.502238 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae065d-5c9f-423b-9840-608f45476b6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:27.503851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae065d-5c9f-423b-9840-608f45476b6b service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:27.513280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1154/2306] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:32:30.757703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad31d03-e3bb-4065-b91f-198c9d9d1233 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:30.759159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fad31d03-e3bb-4065-b91f-198c9d9d1233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:30.775231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad31d03-e3bb-4065-b91f-198c9d9d1233 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:30.780647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:30.781244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:30.781525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:30.781799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:30.783477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:30.843376 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:30.845034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:30.854323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1153/2307] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:32:35.811260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303370b-3721-477d-94d9-2f9cf7c31b6a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:35.811761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303370b-3721-477d-94d9-2f9cf7c31b6a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:35.811846 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303370b-3721-477d-94d9-2f9cf7c31b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:35.832907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1303370b-3721-477d-94d9-2f9cf7c31b6a None None] MFA Rules not processed for user `e7527106829643448f153f9431f233f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:32:35.834192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1303370b-3721-477d-94d9-2f9cf7c31b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:35.841689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1155/2308] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:32:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:32:40.850039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=52 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:32:44.195913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:44.197475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:44.259342 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57da2c0-a38a-4e37-bdca-f54b645d214c service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:44.263030 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:44.263648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:44.264508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:44.264886 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:44.268139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:44.279333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:44.280768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:44.289240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1154/2309] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:32:45.826939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=41 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:32:53.084331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1917ed-8e11-4162-bc38-ef063a396e3b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:53.084909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1917ed-8e11-4162-bc38-ef063a396e3b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:53.085019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1917ed-8e11-4162-bc38-ef063a396e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:53.107615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cb1917ed-8e11-4162-bc38-ef063a396e3b None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:32:53.110173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cb1917ed-8e11-4162-bc38-ef063a396e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:53.169900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1156/2310] 10.222.0.18 () {58 vars in 967 bytes} [Wed Sep 21 05:32:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:32:53.192450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:53.193962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:53.263603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bede38-d9a7-46af-8bbf-ef7e3f53816b service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:53.268704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98852ab-a2e9-4208-9a2e-2502ca79d294 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:53.269314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98852ab-a2e9-4208-9a2e-2502ca79d294 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:53.269596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98852ab-a2e9-4208-9a2e-2502ca79d294 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:53.269871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98852ab-a2e9-4208-9a2e-2502ca79d294 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:53.278635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98852ab-a2e9-4208-9a2e-2502ca79d294 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:53.280201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98852ab-a2e9-4208-9a2e-2502ca79d294 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:53.288121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1155/2311] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:32:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:32:53.383436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1917ed-8e11-4162-bc38-ef063a396e3b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:53.400611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1917ed-8e11-4162-bc38-ef063a396e3b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:53.408135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:53.408793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:53.409067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:53.409370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:53.411328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:53.425048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:53.427122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:53.453027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1157/2312] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:32:57.119144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4d35-88ec-4872-bdb3-4ac0819ad5e0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:57.119144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4d35-88ec-4872-bdb3-4ac0819ad5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:57.119144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4d35-88ec-4872-bdb3-4ac0819ad5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:57.138326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f13e4d35-88ec-4872-bdb3-4ac0819ad5e0 None None] MFA Rules not processed for user `61a257ce292c4c7fa40fe35dbce1964c`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:32:57.139547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f13e4d35-88ec-4872-bdb3-4ac0819ad5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:57.209595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1156/2313] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:32:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:32:57.719395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:57.720876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:57.746630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862af6b-f346-4d54-9418-b7eedef6f83b service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:57.751920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:57.753302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:57.753732 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:57.754938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:57.757369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:57.769302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:57.771894 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:57.782810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1158/2314] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:32:58.249487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4d35-88ec-4872-bdb3-4ac0819ad5e0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:32:58.251108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f13e4d35-88ec-4872-bdb3-4ac0819ad5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:58.304021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:32:58.334433 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4d35-88ec-4872-bdb3-4ac0819ad5e0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:32:58.340413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e1ff3-0959-4420-a671-2777e46e1541 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:32:58.341023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e1ff3-0959-4420-a671-2777e46e1541 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:32:58.341301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e1ff3-0959-4420-a671-2777e46e1541 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:32:58.341571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e1ff3-0959-4420-a671-2777e46e1541 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:32:58.343321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ea7e1ff3-0959-4420-a671-2777e46e1541 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:32:58.354923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e1ff3-0959-4420-a671-2777e46e1541 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:32:58.356596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e1ff3-0959-4420-a671-2777e46e1541 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:32:58.370156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1157/2315] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:32:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:33:01.544047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:01.545631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:01.629592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc9558-7598-4510-b1ce-c7e34601bfa8 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:01.635198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:01.635907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:01.636187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:01.636461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] PATH_INFO: `/v3/users/6073b9546a834ac4bb8801e7ed6eb4db` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:01.637622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6073b9546a834ac4bb8801e7ed6eb4db)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:01.639593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:01.688807 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6073b9546a834ac4bb8801e7ed6eb4db'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:01.689725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6073b9546a834ac4bb8801e7ed6eb4db'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:01.698865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6073b9546a834ac4bb8801e7ed6eb4db'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:01.699478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6073b9546a834ac4bb8801e7ed6eb4db'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:01.707350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6073b9546a834ac4bb8801e7ed6eb4db'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:01.721963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:01.729317 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1159/2316] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:33:01 2022] DELETE /identity/v3/users/6073b9546a834ac4bb8801e7ed6eb4db => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:01.983072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e1ff3-0959-4420-a671-2777e46e1541 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:01.997790 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e1ff3-0959-4420-a671-2777e46e1541 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:02.071929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:02.072558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:02.072834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:02.073105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] PATH_INFO: `/v3/projects/8185f0ff512e4dc192291cc4079ebb2b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:02.074256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8185f0ff512e4dc192291cc4079ebb2b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:02.076151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:02.124330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8185f0ff512e4dc192291cc4079ebb2b', 'request_id': 'req-18784729-2abc-4cff-96e0-554e8dd8a5e5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.131468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8185f0ff512e4dc192291cc4079ebb2b', 'request_id': 'req-18784729-2abc-4cff-96e0-554e8dd8a5e5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.137244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8185f0ff512e4dc192291cc4079ebb2b', 'request_id': 'req-18784729-2abc-4cff-96e0-554e8dd8a5e5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.148332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] The token cache is being invalidate because project 8185f0ff512e4dc192291cc4079ebb2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:33:02.148617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.149235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.150745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1158/2317] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:33:01 2022] DELETE /identity/v3/projects/8185f0ff512e4dc192291cc4079ebb2b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:02.154750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:02.156145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:02.169291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f0aca-b6c7-4861-bcaa-fa64abff9d4e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:02.223843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:02.224397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:02.224669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:02.224936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] PATH_INFO: `/v3/users/34329bbfe65941a8b37b999622d8d9d4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:02.226551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34329bbfe65941a8b37b999622d8d9d4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:02.228481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:02.272299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34329bbfe65941a8b37b999622d8d9d4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.272939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34329bbfe65941a8b37b999622d8d9d4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.280286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34329bbfe65941a8b37b999622d8d9d4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.280903 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34329bbfe65941a8b37b999622d8d9d4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.287747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34329bbfe65941a8b37b999622d8d9d4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.305594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1160/2318] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:33:02 2022] DELETE /identity/v3/users/34329bbfe65941a8b37b999622d8d9d4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:02.491407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:02.493006 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:02.495358 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:02.497092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:02.561704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18784729-2abc-4cff-96e0-554e8dd8a5e5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:02.565518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e05e10-c88a-4abd-8259-1776e77d9d38 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:02.570658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:02.570991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:02.571631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:02.571913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:02.572355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] PATH_INFO: `/v3/projects/db66bf4fdc61433594f173b1f92fab39` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:02.573670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=db66bf4fdc61433594f173b1f92fab39)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:02.575701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:02.604839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:02.621359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:02.621993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:02.622275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:02.622543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:02.624282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:02.632655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db66bf4fdc61433594f173b1f92fab39', 'request_id': 'req-0e695a10-9ac7-4932-81f6-da22f7c397dc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.638280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db66bf4fdc61433594f173b1f92fab39', 'request_id': 'req-0e695a10-9ac7-4932-81f6-da22f7c397dc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.646369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db66bf4fdc61433594f173b1f92fab39', 'request_id': 'req-0e695a10-9ac7-4932-81f6-da22f7c397dc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.659141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] The token cache is being invalidate because project db66bf4fdc61433594f173b1f92fab39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:33:02.659425 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.660046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0e695a10-9ac7-4932-81f6-da22f7c397dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:02.661584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1161/2319] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:33:02 2022] DELETE /identity/v3/projects/db66bf4fdc61433594f173b1f92fab39 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:02.694643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:02.697701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:02.709583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1159/2320] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:33:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:33:09.928303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44066c-a6bd-45bf-8e9f-f6087a77bfcc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:09.928808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44066c-a6bd-45bf-8e9f-f6087a77bfcc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:09.928928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44066c-a6bd-45bf-8e9f-f6087a77bfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:09.947924 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3b44066c-a6bd-45bf-8e9f-f6087a77bfcc None None] MFA Rules not processed for user `9fb2f0174dd04c0fb2aa0ac142ffce95`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:33:09.949135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3b44066c-a6bd-45bf-8e9f-f6087a77bfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:10.002839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1162/2321] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:33:11.674329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:11.675883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:11.752407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2de928-1183-4901-9ae9-400dcdcbb993 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:11.838172 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:11.847116 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:11.847745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:11.848168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:11.848448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:11.850820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:11.865037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:11.867002 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:11.886165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1160/2322] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:33:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:33:19.364380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44066c-a6bd-45bf-8e9f-f6087a77bfcc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:19.366468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3b44066c-a6bd-45bf-8e9f-f6087a77bfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:19.384765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44066c-a6bd-45bf-8e9f-f6087a77bfcc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:19.390283 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:19.391053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:19.391337 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:19.391610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:19.393406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:19.406285 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:19.407915 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:19.420054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1163/2323] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:33:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:33:21.875132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:29.414417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:44.351883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:44.353375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:44.444092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c073b-792d-49da-9657-1c80ddce484e service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:44.448435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e0fe9-5930-4862-a83b-f1586ba50587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:44.450225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e0fe9-5930-4862-a83b-f1586ba50587 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:44.450616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e0fe9-5930-4862-a83b-f1586ba50587 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:44.450993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e0fe9-5930-4862-a83b-f1586ba50587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:44.452959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e80e0fe9-5930-4862-a83b-f1586ba50587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:44.531769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e0fe9-5930-4862-a83b-f1586ba50587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:44.533638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e0fe9-5930-4862-a83b-f1586ba50587 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:44.545563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1161/2324] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:33:48.093776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:48.096486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:48.113044 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e19bcc-3825-44ff-b622-d1961ea7c23f service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:48.119604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:48.120228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.120612 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.120781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] PATH_INFO: `/v3/users/e7527106829643448f153f9431f233f1` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.121969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7527106829643448f153f9431f233f1)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:48.123902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:48.174750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7527106829643448f153f9431f233f1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.175402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7527106829643448f153f9431f233f1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.183814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7527106829643448f153f9431f233f1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.184542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7527106829643448f153f9431f233f1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.191991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ea78ac79-21d8-4d1f-a622-761363e3f16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7527106829643448f153f9431f233f1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.211240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1164/2325] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:33:48 2022] DELETE /identity/v3/users/e7527106829643448f153f9431f233f1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:48.394291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e0fe9-5930-4862-a83b-f1586ba50587 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:48.404391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e0fe9-5930-4862-a83b-f1586ba50587 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:48.469552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:48.470217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.470497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.470770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] PATH_INFO: `/v3/projects/e8fac41f41e5499aae33fd726b61f8a7` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.471878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8fac41f41e5499aae33fd726b61f8a7)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:48.473797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:48.521154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8fac41f41e5499aae33fd726b61f8a7', 'request_id': 'req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.524503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:48.530642 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8fac41f41e5499aae33fd726b61f8a7', 'request_id': 'req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.537227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8fac41f41e5499aae33fd726b61f8a7', 'request_id': 'req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.548979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] The token cache is being invalidate because project e8fac41f41e5499aae33fd726b61f8a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:33:48.549297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.550068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5d923bab-0bfe-49be-9f5b-0c21f1f9657b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:48.552046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1162/2326] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:33:48 2022] DELETE /identity/v3/projects/e8fac41f41e5499aae33fd726b61f8a7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:48.556750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54930-b540-46ab-ac47-3e42e89b2bb6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.557045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54930-b540-46ab-ac47-3e42e89b2bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.557338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54930-b540-46ab-ac47-3e42e89b2bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.578756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b9a54930-b540-46ab-ac47-3e42e89b2bb6 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:33:48.579994 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b9a54930-b540-46ab-ac47-3e42e89b2bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:48.603381 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45c94e-f215-40bf-b019-253ce30c604e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.603715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45c94e-f215-40bf-b019-253ce30c604e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.603948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45c94e-f215-40bf-b019-253ce30c604e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.623294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8c45c94e-f215-40bf-b019-253ce30c604e None None] MFA Rules not processed for user `1ec24729b04947b7b05cc6dcd787eb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:33:48.624504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8c45c94e-f215-40bf-b019-253ce30c604e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:48.645132 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1165/2327] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:33:48.653442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6453-d33d-43f7-bc74-9023cb51bf3d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.653726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6453-d33d-43f7-bc74-9023cb51bf3d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.654036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6453-d33d-43f7-bc74-9023cb51bf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.673235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-25ac6453-d33d-43f7-bc74-9023cb51bf3d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:33:48.675099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-25ac6453-d33d-43f7-bc74-9023cb51bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:48.680841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1163/2328] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:33:48.682948 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1166/2329] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:33:48.687201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45c94e-f215-40bf-b019-253ce30c604e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:48.702922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45c94e-f215-40bf-b019-253ce30c604e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:48.707558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6d65-f063-4908-aa89-0cf68f6a6eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:48.708170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6d65-f063-4908-aa89-0cf68f6a6eed admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.708454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6d65-f063-4908-aa89-0cf68f6a6eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.708729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6d65-f063-4908-aa89-0cf68f6a6eed admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.709378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6d65-f063-4908-aa89-0cf68f6a6eed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:33:48.709691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6d65-f063-4908-aa89-0cf68f6a6eed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:48.711886 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6d65-f063-4908-aa89-0cf68f6a6eed admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:48.724166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1164/2330] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:33:48.729189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac6453-d33d-43f7-bc74-9023cb51bf3d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:48.738714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac6453-d33d-43f7-bc74-9023cb51bf3d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:48.743700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a1ce0-eba6-4774-8b2a-5fd175ad4291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:48.744201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a1ce0-eba6-4774-8b2a-5fd175ad4291 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.744556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a1ce0-eba6-4774-8b2a-5fd175ad4291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.744840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a1ce0-eba6-4774-8b2a-5fd175ad4291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.745467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a1ce0-eba6-4774-8b2a-5fd175ad4291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:48.747405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a1ce0-eba6-4774-8b2a-5fd175ad4291 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:48.777963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1167/2331] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:33:48 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:33:48.781009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6d65-f063-4908-aa89-0cf68f6a6eed admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:48.791799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6d65-f063-4908-aa89-0cf68f6a6eed admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:48.849956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4441f9-6c05-4f4c-a224-f580df10842e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:48.850641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4441f9-6c05-4f4c-a224-f580df10842e admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.850927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4441f9-6c05-4f4c-a224-f580df10842e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.851233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4441f9-6c05-4f4c-a224-f580df10842e admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.851878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4441f9-6c05-4f4c-a224-f580df10842e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:48.853776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4441f9-6c05-4f4c-a224-f580df10842e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:48.876426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1165/2332] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:33:48 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:33:48.880616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a1ce0-eba6-4774-8b2a-5fd175ad4291 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:48.891147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a1ce0-eba6-4774-8b2a-5fd175ad4291 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:48.895687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180feb6-d9cd-4020-ae49-d1fba6d541b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:48.896278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180feb6-d9cd-4020-ae49-d1fba6d541b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.896553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180feb6-d9cd-4020-ae49-d1fba6d541b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.896821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180feb6-d9cd-4020-ae49-d1fba6d541b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.897451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180feb6-d9cd-4020-ae49-d1fba6d541b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:48.899084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180feb6-d9cd-4020-ae49-d1fba6d541b1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:48.909042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1168/2333] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:33:48.912921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4441f9-6c05-4f4c-a224-f580df10842e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:48.922981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4441f9-6c05-4f4c-a224-f580df10842e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:48.927610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa44b8-50a0-4bd9-a494-d75badc3c1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:48.928210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa44b8-50a0-4bd9-a494-d75badc3c1bd admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.928494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa44b8-50a0-4bd9-a494-d75badc3c1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.928771 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa44b8-50a0-4bd9-a494-d75badc3c1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.929961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa44b8-50a0-4bd9-a494-d75badc3c1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:48.931595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa44b8-50a0-4bd9-a494-d75badc3c1bd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:48.943006 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1166/2334] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:33:48.947565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180feb6-d9cd-4020-ae49-d1fba6d541b1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:48.957355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180feb6-d9cd-4020-ae49-d1fba6d541b1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:48.961882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d195a-ae47-4e84-b0d2-22bb48c276ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:48.962777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d195a-ae47-4e84-b0d2-22bb48c276ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:48.963029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d195a-ae47-4e84-b0d2-22bb48c276ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:48.963370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d195a-ae47-4e84-b0d2-22bb48c276ad admin admin] PATH_INFO: `/v3/projects/c777000f5e9f4fb9a8905a326eda12cc/users/741706e21edd4732b966bd9a167849ff/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:48.978679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d195a-ae47-4e84-b0d2-22bb48c276ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=c777000f5e9f4fb9a8905a326eda12cc, user_id=741706e21edd4732b966bd9a167849ff, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:48.981322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d195a-ae47-4e84-b0d2-22bb48c276ad admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:48.996074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1169/2335] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:33:48 2022] PUT /identity/v3/projects/c777000f5e9f4fb9a8905a326eda12cc/users/741706e21edd4732b966bd9a167849ff/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:49.007194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05716b4f-e467-4840-80af-69233fec24d4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:49.008070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05716b4f-e467-4840-80af-69233fec24d4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:49.008345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05716b4f-e467-4840-80af-69233fec24d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:49.036852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-05716b4f-e467-4840-80af-69233fec24d4 None None] MFA Rules not processed for user `741706e21edd4732b966bd9a167849ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:33:49.038098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-05716b4f-e467-4840-80af-69233fec24d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:49.078390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:49.104698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1167/2336] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:33:49.115640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d195a-ae47-4e84-b0d2-22bb48c276ad admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:49.117855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b30d195a-ae47-4e84-b0d2-22bb48c276ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:49.128555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:49.177456 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d195a-ae47-4e84-b0d2-22bb48c276ad admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:49.181688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04229f-8d7d-4be0-97d2-faf60c0d3bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:49.182329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04229f-8d7d-4be0-97d2-faf60c0d3bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:49.182608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04229f-8d7d-4be0-97d2-faf60c0d3bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:49.182880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04229f-8d7d-4be0-97d2-faf60c0d3bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:49.191637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04229f-8d7d-4be0-97d2-faf60c0d3bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:49.193060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04229f-8d7d-4be0-97d2-faf60c0d3bf0 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:49.201796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1170/2337] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:33:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:33:57.660225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05716b4f-e467-4840-80af-69233fec24d4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:57.731241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05716b4f-e467-4840-80af-69233fec24d4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:57.735739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:57.736634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:57.736914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:57.737188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] PATH_INFO: `/v3/users/741706e21edd4732b966bd9a167849ff` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:57.738401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] RBAC: Authorizing `identity:delete_user(user_id=741706e21edd4732b966bd9a167849ff)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:57.740309 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:57.789280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '741706e21edd4732b966bd9a167849ff'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:57.798098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '741706e21edd4732b966bd9a167849ff'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:57.798738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '741706e21edd4732b966bd9a167849ff'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:57.799336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '741706e21edd4732b966bd9a167849ff'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:57.807019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '741706e21edd4732b966bd9a167849ff'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:57.824495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1168/2338] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:33:57 2022] DELETE /identity/v3/users/741706e21edd4732b966bd9a167849ff => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:57.829033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c2184-eea1-4d6e-92c6-5511a5340f84 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:57.829496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c2184-eea1-4d6e-92c6-5511a5340f84 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:57.829715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c2184-eea1-4d6e-92c6-5511a5340f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:57.848192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2f7c2184-eea1-4d6e-92c6-5511a5340f84 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:33:57.849396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2f7c2184-eea1-4d6e-92c6-5511a5340f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:57.907891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1171/2339] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:33:57.940858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:57.942332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:57.998910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31d524-0c2a-4d7c-8e75-86c53318530d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:58.002660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64620043-fbd1-492c-91e9-199482a22ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:58.003285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64620043-fbd1-492c-91e9-199482a22ade service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:58.003542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64620043-fbd1-492c-91e9-199482a22ade service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:58.003815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64620043-fbd1-492c-91e9-199482a22ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:58.013152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64620043-fbd1-492c-91e9-199482a22ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:58.014605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64620043-fbd1-492c-91e9-199482a22ade service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:58.024444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1169/2340] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:33:58.471545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c2184-eea1-4d6e-92c6-5511a5340f84 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:58.472599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2f7c2184-eea1-4d6e-92c6-5511a5340f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:58.487290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c2184-eea1-4d6e-92c6-5511a5340f84 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:58.492328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:58.492929 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:58.493370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:58.493501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] PATH_INFO: `/v3/projects/c777000f5e9f4fb9a8905a326eda12cc` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:58.495254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c777000f5e9f4fb9a8905a326eda12cc)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:58.497924 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:58.548410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c777000f5e9f4fb9a8905a326eda12cc', 'request_id': 'req-f4e61c73-d12c-43ba-a041-69593dea0a2a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:58.554448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c777000f5e9f4fb9a8905a326eda12cc', 'request_id': 'req-f4e61c73-d12c-43ba-a041-69593dea0a2a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:58.563690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c777000f5e9f4fb9a8905a326eda12cc', 'request_id': 'req-f4e61c73-d12c-43ba-a041-69593dea0a2a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:58.576277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] The token cache is being invalidate because project c777000f5e9f4fb9a8905a326eda12cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:33:58.576764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:58.577669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f4e61c73-d12c-43ba-a041-69593dea0a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:33:58.585667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1172/2341] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:33:58 2022] DELETE /identity/v3/projects/c777000f5e9f4fb9a8905a326eda12cc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:58.588010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b564aa-53c4-4b06-b8f8-07c297f080b2 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:58.588252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b564aa-53c4-4b06-b8f8-07c297f080b2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:58.588702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b564aa-53c4-4b06-b8f8-07c297f080b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:58.597187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:58.613103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-08b564aa-53c4-4b06-b8f8-07c297f080b2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:33:58.614415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-08b564aa-53c4-4b06-b8f8-07c297f080b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:58.673067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1170/2342] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:33:58.681140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abe66c-8c20-4897-9007-3ade947c50c3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:58.681447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abe66c-8c20-4897-9007-3ade947c50c3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:58.681722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abe66c-8c20-4897-9007-3ade947c50c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:58.703685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-35abe66c-8c20-4897-9007-3ade947c50c3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:33:58.704934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-35abe66c-8c20-4897-9007-3ade947c50c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:58.713829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1173/2343] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:33:58.718315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b564aa-53c4-4b06-b8f8-07c297f080b2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:58.729294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b564aa-53c4-4b06-b8f8-07c297f080b2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:58.734173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe370c-3013-4205-b220-1a3da81c8226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:58.734835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe370c-3013-4205-b220-1a3da81c8226 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:58.735115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe370c-3013-4205-b220-1a3da81c8226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:58.735387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe370c-3013-4205-b220-1a3da81c8226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:58.736337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe370c-3013-4205-b220-1a3da81c8226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:33:58.736638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe370c-3013-4205-b220-1a3da81c8226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:58.738344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe370c-3013-4205-b220-1a3da81c8226 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:58.751469 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1171/2344] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:33:58.755711 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abe66c-8c20-4897-9007-3ade947c50c3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:58.765714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abe66c-8c20-4897-9007-3ade947c50c3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:58.770639 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75943843-1373-4bd3-a357-5d07ba46cd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:58.771340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75943843-1373-4bd3-a357-5d07ba46cd33 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:58.771682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75943843-1373-4bd3-a357-5d07ba46cd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:58.772017 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75943843-1373-4bd3-a357-5d07ba46cd33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:58.772729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75943843-1373-4bd3-a357-5d07ba46cd33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:58.775375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75943843-1373-4bd3-a357-5d07ba46cd33 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:58.805873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1174/2345] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:33:58 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:33:58.809889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe370c-3013-4205-b220-1a3da81c8226 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:58.822585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe370c-3013-4205-b220-1a3da81c8226 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:58.900341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce374ad1-7688-43f6-8398-aa36b4436b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:58.900967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce374ad1-7688-43f6-8398-aa36b4436b55 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:58.901250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce374ad1-7688-43f6-8398-aa36b4436b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:58.901524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce374ad1-7688-43f6-8398-aa36b4436b55 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:58.902209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce374ad1-7688-43f6-8398-aa36b4436b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:58.904274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce374ad1-7688-43f6-8398-aa36b4436b55 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:58.938481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1172/2346] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:33:58 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:33:58.939770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75943843-1373-4bd3-a357-5d07ba46cd33 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:58.951454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75943843-1373-4bd3-a357-5d07ba46cd33 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:58.957173 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd43a8-0bc1-468f-855c-99fa16f34604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:58.957951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd43a8-0bc1-468f-855c-99fa16f34604 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:58.958093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd43a8-0bc1-468f-855c-99fa16f34604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:58.958343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd43a8-0bc1-468f-855c-99fa16f34604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:58.958979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd43a8-0bc1-468f-855c-99fa16f34604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:58.960585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd43a8-0bc1-468f-855c-99fa16f34604 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:58.962045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:33:58.972490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1175/2347] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:33:58.976408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce374ad1-7688-43f6-8398-aa36b4436b55 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:58.987838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce374ad1-7688-43f6-8398-aa36b4436b55 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:58.993554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85ccc9-3ce7-4d10-b356-a943f97bfa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:58.994284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85ccc9-3ce7-4d10-b356-a943f97bfa13 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:58.994630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85ccc9-3ce7-4d10-b356-a943f97bfa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:58.994973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85ccc9-3ce7-4d10-b356-a943f97bfa13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:58.995874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85ccc9-3ce7-4d10-b356-a943f97bfa13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:58.997534 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85ccc9-3ce7-4d10-b356-a943f97bfa13 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:59.009899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1173/2348] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:33:59.014019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd43a8-0bc1-468f-855c-99fa16f34604 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:59.025160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd43a8-0bc1-468f-855c-99fa16f34604 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:59.030449 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e99e9d-ddd0-4282-b1b5-4cda512e27a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:59.031184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e99e9d-ddd0-4282-b1b5-4cda512e27a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:59.031541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e99e9d-ddd0-4282-b1b5-4cda512e27a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:59.032060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e99e9d-ddd0-4282-b1b5-4cda512e27a2 admin admin] PATH_INFO: `/v3/projects/cd59e0e21003427daea955bcd0dc6bff/users/e081f4cbd62042ebab329819a43cd858/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:59.049491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e99e9d-ddd0-4282-b1b5-4cda512e27a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd59e0e21003427daea955bcd0dc6bff, user_id=e081f4cbd62042ebab329819a43cd858, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:59.053149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e99e9d-ddd0-4282-b1b5-4cda512e27a2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:59.075861 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1176/2349] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:33:59 2022] PUT /identity/v3/projects/cd59e0e21003427daea955bcd0dc6bff/users/e081f4cbd62042ebab329819a43cd858/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:33:59.080966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475f6d0-e9df-4b1e-91b3-1b7f2b7ed312 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:59.081251 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475f6d0-e9df-4b1e-91b3-1b7f2b7ed312 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:59.081690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475f6d0-e9df-4b1e-91b3-1b7f2b7ed312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:59.102048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c475f6d0-e9df-4b1e-91b3-1b7f2b7ed312 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:33:59.103240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c475f6d0-e9df-4b1e-91b3-1b7f2b7ed312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:59.170267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1174/2350] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:33:59.180060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e99e9d-ddd0-4282-b1b5-4cda512e27a2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:33:59.182411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c9e99e9d-ddd0-4282-b1b5-4cda512e27a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:33:59.251320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e99e9d-ddd0-4282-b1b5-4cda512e27a2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:33:59.255146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae327ea-fc11-43f6-a3f0-fbdfa3a6337e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:33:59.255752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae327ea-fc11-43f6-a3f0-fbdfa3a6337e service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:59.256194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae327ea-fc11-43f6-a3f0-fbdfa3a6337e service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:59.256498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae327ea-fc11-43f6-a3f0-fbdfa3a6337e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:33:59.267652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae327ea-fc11-43f6-a3f0-fbdfa3a6337e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:33:59.269092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae327ea-fc11-43f6-a3f0-fbdfa3a6337e service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:33:59.279493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1177/2351] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:33:59.981731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f620a8-24ba-44e3-9181-a83633712204 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:33:59.982306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f620a8-24ba-44e3-9181-a83633712204 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:33:59.982626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f620a8-24ba-44e3-9181-a83633712204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:00.011918 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:00.029246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e3f620a8-24ba-44e3-9181-a83633712204 None None] MFA Rules not processed for user `e081f4cbd62042ebab329819a43cd858`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:00.030629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e3f620a8-24ba-44e3-9181-a83633712204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:00.112734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae327ea-fc11-43f6-a3f0-fbdfa3a6337e service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:00.123736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae327ea-fc11-43f6-a3f0-fbdfa3a6337e service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:00.133109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7347f-54de-4e09-9cc1-81d675d852da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:00.134949 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7347f-54de-4e09-9cc1-81d675d852da service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:00.135658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7347f-54de-4e09-9cc1-81d675d852da service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:00.135989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7347f-54de-4e09-9cc1-81d675d852da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:00.139212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0ec7347f-54de-4e09-9cc1-81d675d852da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:00.142120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1175/2352] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:33:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:00.147624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f620a8-24ba-44e3-9181-a83633712204 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:00.169340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f620a8-24ba-44e3-9181-a83633712204 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:00.178378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cc60f-6eb5-476a-9019-7567057dc952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:00.179013 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cc60f-6eb5-476a-9019-7567057dc952 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:00.179297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cc60f-6eb5-476a-9019-7567057dc952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:00.179577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cc60f-6eb5-476a-9019-7567057dc952 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:00.180227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cc60f-6eb5-476a-9019-7567057dc952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:00.182769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cc60f-6eb5-476a-9019-7567057dc952 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:00.221308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1176/2353] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:00 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:00.224950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cc60f-6eb5-476a-9019-7567057dc952 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:00.244461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7347f-54de-4e09-9cc1-81d675d852da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:00.247100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7347f-54de-4e09-9cc1-81d675d852da service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:00.260029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1178/2354] 10.222.0.18 () {60 vars in 1396 bytes} [Wed Sep 21 05:34:00 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:34:00.316373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cc60f-6eb5-476a-9019-7567057dc952 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:00.322765 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dc329-927a-429a-b112-7df28cca6934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:00.323100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dc329-927a-429a-b112-7df28cca6934 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:00.323544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dc329-927a-429a-b112-7df28cca6934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:00.323963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dc329-927a-429a-b112-7df28cca6934 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:00.324762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6dc329-927a-429a-b112-7df28cca6934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:00.326868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6dc329-927a-429a-b112-7df28cca6934 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:00.374258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1177/2355] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:00 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:00.375260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7347f-54de-4e09-9cc1-81d675d852da service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:00.390589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7347f-54de-4e09-9cc1-81d675d852da service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:00.396181 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc196b-9afd-4a53-a60c-63093f22a97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:00.397086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc196b-9afd-4a53-a60c-63093f22a97a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:00.397477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc196b-9afd-4a53-a60c-63093f22a97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:00.397849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc196b-9afd-4a53-a60c-63093f22a97a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:00.398654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc196b-9afd-4a53-a60c-63093f22a97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:00.401133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc196b-9afd-4a53-a60c-63093f22a97a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:00.413736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1179/2356] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:34:00.420081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dc329-927a-429a-b112-7df28cca6934 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:00.433981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dc329-927a-429a-b112-7df28cca6934 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:00.438647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237233f-c024-4346-b627-f482d89e8171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:00.439301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237233f-c024-4346-b627-f482d89e8171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:00.439571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237233f-c024-4346-b627-f482d89e8171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:00.439845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237233f-c024-4346-b627-f482d89e8171 admin admin] PATH_INFO: `/v3/projects/1dd5691733514d80829946f9c31ab294/users/fa0d0852358c437980a3a003ce10f65c/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:00.456415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237233f-c024-4346-b627-f482d89e8171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd5691733514d80829946f9c31ab294, user_id=fa0d0852358c437980a3a003ce10f65c, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:00.459297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237233f-c024-4346-b627-f482d89e8171 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:00.473590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1178/2357] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:00 2022] PUT /identity/v3/projects/1dd5691733514d80829946f9c31ab294/users/fa0d0852358c437980a3a003ce10f65c/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:00.478631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc196b-9afd-4a53-a60c-63093f22a97a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:00.489136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc196b-9afd-4a53-a60c-63093f22a97a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:00.547763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04f629-e6d5-4896-84e5-41ee7c39e947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:00.548730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04f629-e6d5-4896-84e5-41ee7c39e947 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:00.548730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04f629-e6d5-4896-84e5-41ee7c39e947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:00.548904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04f629-e6d5-4896-84e5-41ee7c39e947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:00.549618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04f629-e6d5-4896-84e5-41ee7c39e947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:00.551475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04f629-e6d5-4896-84e5-41ee7c39e947 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:00.553485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:00.564976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1180/2358] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:00.569477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237233f-c024-4346-b627-f482d89e8171 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:00.580668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237233f-c024-4346-b627-f482d89e8171 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:00.585448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4cfcd-6477-456a-9f74-2d356b395ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:00.586137 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4cfcd-6477-456a-9f74-2d356b395ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:00.586420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4cfcd-6477-456a-9f74-2d356b395ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:00.587276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4cfcd-6477-456a-9f74-2d356b395ea4 admin admin] PATH_INFO: `/v3/projects/1dd5691733514d80829946f9c31ab294/users/fa0d0852358c437980a3a003ce10f65c/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:00.589192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4cfcd-6477-456a-9f74-2d356b395ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd5691733514d80829946f9c31ab294, user_id=fa0d0852358c437980a3a003ce10f65c, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:00.591984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4cfcd-6477-456a-9f74-2d356b395ea4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:00.603550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1179/2359] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:00 2022] PUT /identity/v3/projects/1dd5691733514d80829946f9c31ab294/users/fa0d0852358c437980a3a003ce10f65c/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:01.211973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e773fa-b219-4de7-ba70-8e161d71303b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:01.212505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e773fa-b219-4de7-ba70-8e161d71303b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:01.212505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e773fa-b219-4de7-ba70-8e161d71303b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:01.241413 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e7e773fa-b219-4de7-ba70-8e161d71303b None None] MFA Rules not processed for user `fa0d0852358c437980a3a003ce10f65c`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:01.242767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e7e773fa-b219-4de7-ba70-8e161d71303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:01.316182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1181/2360] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:01.328530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4cfcd-6477-456a-9f74-2d356b395ea4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:01.330331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-28a4cfcd-6477-456a-9f74-2d356b395ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:01.365558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:01.415378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4cfcd-6477-456a-9f74-2d356b395ea4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:01.418994 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f26880-d956-4410-b127-eac2216fcfa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:01.419594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f26880-d956-4410-b127-eac2216fcfa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:01.419874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f26880-d956-4410-b127-eac2216fcfa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:01.420147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f26880-d956-4410-b127-eac2216fcfa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:01.484714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f26880-d956-4410-b127-eac2216fcfa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:01.486565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f26880-d956-4410-b127-eac2216fcfa2 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:01.496762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1180/2361] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:02.888392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e773fa-b219-4de7-ba70-8e161d71303b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:02.889800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e7e773fa-b219-4de7-ba70-8e161d71303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:02.901600 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e773fa-b219-4de7-ba70-8e161d71303b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:02.905187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410ce74-3a0f-4c68-9cc6-ab6abb1c0f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:02.905784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410ce74-3a0f-4c68-9cc6-ab6abb1c0f19 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:02.906109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410ce74-3a0f-4c68-9cc6-ab6abb1c0f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:02.906388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410ce74-3a0f-4c68-9cc6-ab6abb1c0f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:02.915065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410ce74-3a0f-4c68-9cc6-ab6abb1c0f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:02.916484 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410ce74-3a0f-4c68-9cc6-ab6abb1c0f19 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:02.925667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1182/2362] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:10.997176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca56828-08b1-4ffc-8dfe-f98c58a97818 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:10.997639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca56828-08b1-4ffc-8dfe-f98c58a97818 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:10.997876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca56828-08b1-4ffc-8dfe-f98c58a97818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:11.025748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fca56828-08b1-4ffc-8dfe-f98c58a97818 None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:11.027067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fca56828-08b1-4ffc-8dfe-f98c58a97818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:11.136388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1181/2363] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:34:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:11.201351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410ce74-3a0f-4c68-9cc6-ab6abb1c0f19 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:11.202849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e410ce74-3a0f-4c68-9cc6-ab6abb1c0f19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:11.281287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410ce74-3a0f-4c68-9cc6-ab6abb1c0f19 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:11.286385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70295d-f15e-4674-be29-2f321f5c37cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:11.286977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70295d-f15e-4674-be29-2f321f5c37cc service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:11.287258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70295d-f15e-4674-be29-2f321f5c37cc service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:11.287557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70295d-f15e-4674-be29-2f321f5c37cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:11.296762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70295d-f15e-4674-be29-2f321f5c37cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:11.298571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70295d-f15e-4674-be29-2f321f5c37cc service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:11.307924 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1183/2364] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:34:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:34:16.847409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca56828-08b1-4ffc-8dfe-f98c58a97818 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:16.920601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca56828-08b1-4ffc-8dfe-f98c58a97818 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:16.925430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:16.926111 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:16.926395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:16.926671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] PATH_INFO: `/v3/users/e081f4cbd62042ebab329819a43cd858` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:16.927832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e081f4cbd62042ebab329819a43cd858)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:16.929797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:16.971471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:16.987768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e081f4cbd62042ebab329819a43cd858'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.001982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e081f4cbd62042ebab329819a43cd858'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.001982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e081f4cbd62042ebab329819a43cd858'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.001982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e081f4cbd62042ebab329819a43cd858'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.009882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e081f4cbd62042ebab329819a43cd858'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.032687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1182/2365] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:16 2022] DELETE /identity/v3/users/e081f4cbd62042ebab329819a43cd858 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:17.347072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70295d-f15e-4674-be29-2f321f5c37cc service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:17.361995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70295d-f15e-4674-be29-2f321f5c37cc service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:17.455264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:17.455921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:17.456207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:17.456482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] PATH_INFO: `/v3/projects/cd59e0e21003427daea955bcd0dc6bff` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:17.457688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd59e0e21003427daea955bcd0dc6bff)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:17.460318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:17.462012 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:17.526091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd59e0e21003427daea955bcd0dc6bff', 'request_id': 'req-607aef03-f095-4071-92ca-c19ecc01de33'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.535443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd59e0e21003427daea955bcd0dc6bff', 'request_id': 'req-607aef03-f095-4071-92ca-c19ecc01de33'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.544452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd59e0e21003427daea955bcd0dc6bff', 'request_id': 'req-607aef03-f095-4071-92ca-c19ecc01de33'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.558810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] The token cache is being invalidate because project cd59e0e21003427daea955bcd0dc6bff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:17.559151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.559836 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.561360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1184/2366] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:17 2022] DELETE /identity/v3/projects/cd59e0e21003427daea955bcd0dc6bff => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:17.565792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:17.569461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:17.596361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2067-8d84-4d1e-9e62-64f3cfdc2965 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:17.673749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:17.674625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:17.674938 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:17.675197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] PATH_INFO: `/v3/users/fa0d0852358c437980a3a003ce10f65c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:17.677718 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa0d0852358c437980a3a003ce10f65c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:17.680366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:17.750792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa0d0852358c437980a3a003ce10f65c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.762049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa0d0852358c437980a3a003ce10f65c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.763417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa0d0852358c437980a3a003ce10f65c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.764362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa0d0852358c437980a3a003ce10f65c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.774215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3edc1ee-d850-43fd-9f10-f1cfb42f9878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa0d0852358c437980a3a003ce10f65c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:17.798187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1183/2367] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:17 2022] DELETE /identity/v3/users/fa0d0852358c437980a3a003ce10f65c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:18.032570 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:18.034170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:18.055760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aef03-f095-4071-92ca-c19ecc01de33 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:18.149041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:18.149529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.149783 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.150135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] PATH_INFO: `/v3/projects/1dd5691733514d80829946f9c31ab294` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.151267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd5691733514d80829946f9c31ab294)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:18.153194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:18.211444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd5691733514d80829946f9c31ab294', 'request_id': 'req-dc52470f-679f-411e-8c5b-34c077dfb8aa'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:18.222257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd5691733514d80829946f9c31ab294', 'request_id': 'req-dc52470f-679f-411e-8c5b-34c077dfb8aa'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:18.231707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd5691733514d80829946f9c31ab294', 'request_id': 'req-dc52470f-679f-411e-8c5b-34c077dfb8aa'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:18.251640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] The token cache is being invalidate because project 1dd5691733514d80829946f9c31ab294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:18.252244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:18.253150 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dc52470f-679f-411e-8c5b-34c077dfb8aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:18.258803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1185/2368] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:18 2022] DELETE /identity/v3/projects/1dd5691733514d80829946f9c31ab294 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:18.263860 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbe85c-b5a6-4f86-8cc4-3ae4ffecd895 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.264363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbe85c-b5a6-4f86-8cc4-3ae4ffecd895 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.264476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbe85c-b5a6-4f86-8cc4-3ae4ffecd895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.290052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c7dbe85c-b5a6-4f86-8cc4-3ae4ffecd895 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:18.291861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c7dbe85c-b5a6-4f86-8cc4-3ae4ffecd895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:18.315231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:18.369967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1184/2369] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:18.376953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dd154-10e3-4cc8-bafc-e0aa0dfe84ba None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.377452 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dd154-10e3-4cc8-bafc-e0aa0dfe84ba None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.377676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dd154-10e3-4cc8-bafc-e0aa0dfe84ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.399228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c76dd154-10e3-4cc8-bafc-e0aa0dfe84ba None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:18.401326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c76dd154-10e3-4cc8-bafc-e0aa0dfe84ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:18.409509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1186/2370] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:18.413920 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbe85c-b5a6-4f86-8cc4-3ae4ffecd895 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:18.424502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbe85c-b5a6-4f86-8cc4-3ae4ffecd895 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:18.429099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b72e9-e8b8-4463-826d-b66e2f775f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:18.429703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b72e9-e8b8-4463-826d-b66e2f775f80 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.430009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b72e9-e8b8-4463-826d-b66e2f775f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.430286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b72e9-e8b8-4463-826d-b66e2f775f80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.430979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b72e9-e8b8-4463-826d-b66e2f775f80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:34:18.431285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b72e9-e8b8-4463-826d-b66e2f775f80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:18.432898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b72e9-e8b8-4463-826d-b66e2f775f80 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:18.448349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1185/2371] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:34:18.449647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dd154-10e3-4cc8-bafc-e0aa0dfe84ba None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:18.461811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dd154-10e3-4cc8-bafc-e0aa0dfe84ba None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:18.469659 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8616ce9-2769-46fe-8280-09e1f3abb13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:18.470549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8616ce9-2769-46fe-8280-09e1f3abb13d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.470934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8616ce9-2769-46fe-8280-09e1f3abb13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.471326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8616ce9-2769-46fe-8280-09e1f3abb13d admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.474209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8616ce9-2769-46fe-8280-09e1f3abb13d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:18.474209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8616ce9-2769-46fe-8280-09e1f3abb13d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:18.488300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:18.527529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1187/2372] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:18 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:18.528302 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b72e9-e8b8-4463-826d-b66e2f775f80 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:18.544674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b72e9-e8b8-4463-826d-b66e2f775f80 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:18.633436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bd368-eb92-4ca7-b5ee-731dae404665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:18.634416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bd368-eb92-4ca7-b5ee-731dae404665 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.634732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bd368-eb92-4ca7-b5ee-731dae404665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.635067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bd368-eb92-4ca7-b5ee-731dae404665 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.635952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bd368-eb92-4ca7-b5ee-731dae404665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:18.650234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bd368-eb92-4ca7-b5ee-731dae404665 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:18.670223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1186/2373] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:18 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:18.676455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8616ce9-2769-46fe-8280-09e1f3abb13d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:18.687245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8616ce9-2769-46fe-8280-09e1f3abb13d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:18.694875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9defb94-5d02-461f-82c3-cccffe751e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:18.695656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9defb94-5d02-461f-82c3-cccffe751e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.695935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9defb94-5d02-461f-82c3-cccffe751e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.696210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9defb94-5d02-461f-82c3-cccffe751e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.696904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9defb94-5d02-461f-82c3-cccffe751e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:18.698585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9defb94-5d02-461f-82c3-cccffe751e54 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:18.710476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1188/2374] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:18.717312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bd368-eb92-4ca7-b5ee-731dae404665 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:18.731229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bd368-eb92-4ca7-b5ee-731dae404665 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:18.739437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f187f69-fd17-4fea-9f6a-179f9139f5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:18.739437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f187f69-fd17-4fea-9f6a-179f9139f5cf admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.739437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f187f69-fd17-4fea-9f6a-179f9139f5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.739437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f187f69-fd17-4fea-9f6a-179f9139f5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.739860 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f187f69-fd17-4fea-9f6a-179f9139f5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:18.742770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f187f69-fd17-4fea-9f6a-179f9139f5cf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:18.758748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1187/2375] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:18.760755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9defb94-5d02-461f-82c3-cccffe751e54 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:18.786357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9defb94-5d02-461f-82c3-cccffe751e54 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:18.799049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d783ba-2529-4aff-b93e-d0ff6e62f743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:18.799832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d783ba-2529-4aff-b93e-d0ff6e62f743 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.800216 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d783ba-2529-4aff-b93e-d0ff6e62f743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.801982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d783ba-2529-4aff-b93e-d0ff6e62f743 admin admin] PATH_INFO: `/v3/projects/375748f9acb744e2aa4fdde5c8de4ab5/users/b763c185aeaf44e2b0e4df3d20ac0d28/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.821345 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d783ba-2529-4aff-b93e-d0ff6e62f743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375748f9acb744e2aa4fdde5c8de4ab5, user_id=b763c185aeaf44e2b0e4df3d20ac0d28, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:18.825196 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d783ba-2529-4aff-b93e-d0ff6e62f743 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:18.858170 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1189/2376] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:18 2022] PUT /identity/v3/projects/375748f9acb744e2aa4fdde5c8de4ab5/users/b763c185aeaf44e2b0e4df3d20ac0d28/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:18.858276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cf917-e4b0-46ff-a57f-5c43d0c86c4a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:18.858276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cf917-e4b0-46ff-a57f-5c43d0c86c4a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:18.858444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cf917-e4b0-46ff-a57f-5c43d0c86c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:18.894556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fe6cf917-e4b0-46ff-a57f-5c43d0c86c4a None None] MFA Rules not processed for user `b763c185aeaf44e2b0e4df3d20ac0d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:18.896099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fe6cf917-e4b0-46ff-a57f-5c43d0c86c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:18.992859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1188/2377] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:19.005967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d783ba-2529-4aff-b93e-d0ff6e62f743 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:19.006506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-92d783ba-2529-4aff-b93e-d0ff6e62f743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:19.085319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d783ba-2529-4aff-b93e-d0ff6e62f743 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:19.090038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c964964-a3f5-41ef-87dd-14ad7c2f5c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:19.090745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c964964-a3f5-41ef-87dd-14ad7c2f5c0f service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:19.090915 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c964964-a3f5-41ef-87dd-14ad7c2f5c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:19.091178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c964964-a3f5-41ef-87dd-14ad7c2f5c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:19.102596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c964964-a3f5-41ef-87dd-14ad7c2f5c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:19.104093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c964964-a3f5-41ef-87dd-14ad7c2f5c0f service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:19.114914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1190/2378] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:19.144125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cf917-e4b0-46ff-a57f-5c43d0c86c4a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:19.179801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:19.232963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cf917-e4b0-46ff-a57f-5c43d0c86c4a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:19.240911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:19.241711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:19.242140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:19.242592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] PATH_INFO: `/v3/users/b763c185aeaf44e2b0e4df3d20ac0d28` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:19.244388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b763c185aeaf44e2b0e4df3d20ac0d28)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:19.247427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:19.321000 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b763c185aeaf44e2b0e4df3d20ac0d28'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:19.331654 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b763c185aeaf44e2b0e4df3d20ac0d28'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:19.332467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b763c185aeaf44e2b0e4df3d20ac0d28'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:19.333547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b763c185aeaf44e2b0e4df3d20ac0d28'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:19.345466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b763c185aeaf44e2b0e4df3d20ac0d28'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:19.370257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1189/2379] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:19 2022] DELETE /identity/v3/users/b763c185aeaf44e2b0e4df3d20ac0d28 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:19.374900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0853c77-88f1-4471-9191-db155fbed8b5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:19.375645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0853c77-88f1-4471-9191-db155fbed8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:19.376074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0853c77-88f1-4471-9191-db155fbed8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:19.404058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d0853c77-88f1-4471-9191-db155fbed8b5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:19.406736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d0853c77-88f1-4471-9191-db155fbed8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:19.505357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1191/2380] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:19.515933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:19.517506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:19.580741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d7e5a-bc66-4f30-8b4d-1873ca201b35 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:19.588722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4e593-6a4f-4e77-a46c-ceede72805a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:19.589703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4e593-6a4f-4e77-a46c-ceede72805a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:19.589896 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4e593-6a4f-4e77-a46c-ceede72805a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:19.590296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4e593-6a4f-4e77-a46c-ceede72805a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:19.604794 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4e593-6a4f-4e77-a46c-ceede72805a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:19.606634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4e593-6a4f-4e77-a46c-ceede72805a3 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:19.617950 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1190/2381] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:19.999131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0853c77-88f1-4471-9191-db155fbed8b5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:20.000669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d0853c77-88f1-4471-9191-db155fbed8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:20.014071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0853c77-88f1-4471-9191-db155fbed8b5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:20.019081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:20.019744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.020172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.020457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] PATH_INFO: `/v3/projects/375748f9acb744e2aa4fdde5c8de4ab5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.022052 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375748f9acb744e2aa4fdde5c8de4ab5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:20.023688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:20.076416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375748f9acb744e2aa4fdde5c8de4ab5', 'request_id': 'req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:20.080801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:20.084877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375748f9acb744e2aa4fdde5c8de4ab5', 'request_id': 'req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:20.105828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375748f9acb744e2aa4fdde5c8de4ab5', 'request_id': 'req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:20.119384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] The token cache is being invalidate because project 375748f9acb744e2aa4fdde5c8de4ab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:20.119656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:20.120334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d6cf8efe-a9ca-492a-b2eb-743ff0f05053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:20.121898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1192/2382] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:19 2022] DELETE /identity/v3/projects/375748f9acb744e2aa4fdde5c8de4ab5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:20.127735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d22069-d4cf-4989-b268-eac2d9187e6c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.128109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d22069-d4cf-4989-b268-eac2d9187e6c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.128526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d22069-d4cf-4989-b268-eac2d9187e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.148902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-59d22069-d4cf-4989-b268-eac2d9187e6c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:20.150970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-59d22069-d4cf-4989-b268-eac2d9187e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:20.203661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:20.224520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1191/2383] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:20.233009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d63d5f-59c3-4d13-a579-42fe077165bb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.233299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d63d5f-59c3-4d13-a579-42fe077165bb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.233578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d63d5f-59c3-4d13-a579-42fe077165bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.254503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-10d63d5f-59c3-4d13-a579-42fe077165bb None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:20.255705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-10d63d5f-59c3-4d13-a579-42fe077165bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:20.281927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1193/2384] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:20.285849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d22069-d4cf-4989-b268-eac2d9187e6c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:20.298844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d22069-d4cf-4989-b268-eac2d9187e6c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:20.304100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947511b-aefc-4477-b0fe-0579cbe9011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:20.304716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947511b-aefc-4477-b0fe-0579cbe9011b admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.305005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947511b-aefc-4477-b0fe-0579cbe9011b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.305279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947511b-aefc-4477-b0fe-0579cbe9011b admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.306578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947511b-aefc-4477-b0fe-0579cbe9011b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:34:20.306858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947511b-aefc-4477-b0fe-0579cbe9011b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:20.308725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947511b-aefc-4477-b0fe-0579cbe9011b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:20.324024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1192/2385] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:34:20.328356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d63d5f-59c3-4d13-a579-42fe077165bb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:20.338208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d63d5f-59c3-4d13-a579-42fe077165bb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:20.343846 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85dc80-e84b-45ae-8484-2277199f56d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:20.344895 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85dc80-e84b-45ae-8484-2277199f56d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.345425 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85dc80-e84b-45ae-8484-2277199f56d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.345866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85dc80-e84b-45ae-8484-2277199f56d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.346685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85dc80-e84b-45ae-8484-2277199f56d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:20.348573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85dc80-e84b-45ae-8484-2277199f56d7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:20.392881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1194/2386] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:20 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:20.394025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947511b-aefc-4477-b0fe-0579cbe9011b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:20.404374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947511b-aefc-4477-b0fe-0579cbe9011b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:20.478951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6983e-a0ff-4c3f-8347-15997382214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:20.479990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6983e-a0ff-4c3f-8347-15997382214b admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.480502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6983e-a0ff-4c3f-8347-15997382214b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.481019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6983e-a0ff-4c3f-8347-15997382214b admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.482542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6983e-a0ff-4c3f-8347-15997382214b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:20.486888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6983e-a0ff-4c3f-8347-15997382214b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:20.535034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1193/2387] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:20 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:20.539477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85dc80-e84b-45ae-8484-2277199f56d7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:20.584713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85dc80-e84b-45ae-8484-2277199f56d7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:20.614431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00abe45-1466-410b-b4da-b2a9baa7e88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:20.615388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00abe45-1466-410b-b4da-b2a9baa7e88b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.615852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00abe45-1466-410b-b4da-b2a9baa7e88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.616334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00abe45-1466-410b-b4da-b2a9baa7e88b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.617219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00abe45-1466-410b-b4da-b2a9baa7e88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:20.621307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00abe45-1466-410b-b4da-b2a9baa7e88b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:20.641818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1195/2388] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:20.646020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6983e-a0ff-4c3f-8347-15997382214b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:20.662827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6983e-a0ff-4c3f-8347-15997382214b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:20.669369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb40384-86fc-4f75-8b95-9b6bab84ad7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:20.669711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40384-86fc-4f75-8b95-9b6bab84ad7a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.670023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40384-86fc-4f75-8b95-9b6bab84ad7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.670299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40384-86fc-4f75-8b95-9b6bab84ad7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.670944 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb40384-86fc-4f75-8b95-9b6bab84ad7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:20.672570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb40384-86fc-4f75-8b95-9b6bab84ad7a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:20.702949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1194/2389] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:20.706883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00abe45-1466-410b-b4da-b2a9baa7e88b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:20.720886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00abe45-1466-410b-b4da-b2a9baa7e88b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:20.726484 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a2494-325a-4705-b875-89c012daf34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:20.727379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a2494-325a-4705-b875-89c012daf34a admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.728451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a2494-325a-4705-b875-89c012daf34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.729011 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a2494-325a-4705-b875-89c012daf34a admin admin] PATH_INFO: `/v3/projects/ab07c232383e444a9c75ab425c6a6fb3/users/2fa5b99453054be096f649fa18c029aa/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.750446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a2494-325a-4705-b875-89c012daf34a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab07c232383e444a9c75ab425c6a6fb3, user_id=2fa5b99453054be096f649fa18c029aa, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:20.753847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a2494-325a-4705-b875-89c012daf34a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:20.777973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1196/2390] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:20 2022] PUT /identity/v3/projects/ab07c232383e444a9c75ab425c6a6fb3/users/2fa5b99453054be096f649fa18c029aa/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:20.787513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97674571-f6fa-4ef7-9830-921249121043 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:20.787803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97674571-f6fa-4ef7-9830-921249121043 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:20.788082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97674571-f6fa-4ef7-9830-921249121043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:20.830465 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-97674571-f6fa-4ef7-9830-921249121043 None None] MFA Rules not processed for user `2fa5b99453054be096f649fa18c029aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:20.832473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-97674571-f6fa-4ef7-9830-921249121043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:20.972636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1195/2391] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:20.990321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a2494-325a-4705-b875-89c012daf34a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.072924 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a2494-325a-4705-b875-89c012daf34a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.078155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4b9ee-dbc7-44f6-a765-c69ec32d88ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.078772 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4b9ee-dbc7-44f6-a765-c69ec32d88ef admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.079049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4b9ee-dbc7-44f6-a765-c69ec32d88ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.079333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4b9ee-dbc7-44f6-a765-c69ec32d88ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.079999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4b9ee-dbc7-44f6-a765-c69ec32d88ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.081921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4b9ee-dbc7-44f6-a765-c69ec32d88ef admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.113978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1197/2392] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:20 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:21.115889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97674571-f6fa-4ef7-9830-921249121043 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.127752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97674571-f6fa-4ef7-9830-921249121043 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.169476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:21.190437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25141d-543c-40a3-ac70-d651eb97d5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.191046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25141d-543c-40a3-ac70-d651eb97d5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.191324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25141d-543c-40a3-ac70-d651eb97d5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.191594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25141d-543c-40a3-ac70-d651eb97d5f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.192215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25141d-543c-40a3-ac70-d651eb97d5f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.194143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25141d-543c-40a3-ac70-d651eb97d5f6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.215911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1196/2393] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:21 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:21.220652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4b9ee-dbc7-44f6-a765-c69ec32d88ef admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.237040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4b9ee-dbc7-44f6-a765-c69ec32d88ef admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.242954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f93d7-3c8e-4d9b-b526-bee1fd2cbe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.244222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f93d7-3c8e-4d9b-b526-bee1fd2cbe92 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.244507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f93d7-3c8e-4d9b-b526-bee1fd2cbe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.244784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f93d7-3c8e-4d9b-b526-bee1fd2cbe92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.245431 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f93d7-3c8e-4d9b-b526-bee1fd2cbe92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.247072 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f93d7-3c8e-4d9b-b526-bee1fd2cbe92 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.257993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1198/2394] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:21.261892 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25141d-543c-40a3-ac70-d651eb97d5f6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.272168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25141d-543c-40a3-ac70-d651eb97d5f6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.276767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04739a9-ed31-47f1-91c7-380a3728e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.277369 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04739a9-ed31-47f1-91c7-380a3728e4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.277646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04739a9-ed31-47f1-91c7-380a3728e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.277922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04739a9-ed31-47f1-91c7-380a3728e4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.278586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04739a9-ed31-47f1-91c7-380a3728e4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.280158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04739a9-ed31-47f1-91c7-380a3728e4b0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.291212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1197/2395] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:21.295193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f93d7-3c8e-4d9b-b526-bee1fd2cbe92 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.305289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f93d7-3c8e-4d9b-b526-bee1fd2cbe92 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.310596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d565ff-c9ae-4959-b5f1-046344047acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.311243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d565ff-c9ae-4959-b5f1-046344047acf admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.311524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d565ff-c9ae-4959-b5f1-046344047acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.311799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d565ff-c9ae-4959-b5f1-046344047acf admin admin] PATH_INFO: `/v3/projects/944a1bc5b12a475d81ba90b4df8e7c83/users/4036c8b2d4054027a9817fc06757d807/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.325998 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d565ff-c9ae-4959-b5f1-046344047acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=944a1bc5b12a475d81ba90b4df8e7c83, user_id=4036c8b2d4054027a9817fc06757d807, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.328646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d565ff-c9ae-4959-b5f1-046344047acf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.341668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1199/2396] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:21 2022] PUT /identity/v3/projects/944a1bc5b12a475d81ba90b4df8e7c83/users/4036c8b2d4054027a9817fc06757d807/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:21.351835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3b833-84ff-4934-98bf-244ad5426b5f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.352300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3b833-84ff-4934-98bf-244ad5426b5f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.352420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3b833-84ff-4934-98bf-244ad5426b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.381389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-dba3b833-84ff-4934-98bf-244ad5426b5f None None] MFA Rules not processed for user `4036c8b2d4054027a9817fc06757d807`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:21.382644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dba3b833-84ff-4934-98bf-244ad5426b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:21.447558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1198/2397] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:21.452155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d565ff-c9ae-4959-b5f1-046344047acf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.483860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:21.515515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d565ff-c9ae-4959-b5f1-046344047acf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.520497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f174c-c252-41aa-8712-afadd51eda3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.521093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f174c-c252-41aa-8712-afadd51eda3f admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.521372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f174c-c252-41aa-8712-afadd51eda3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.521641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f174c-c252-41aa-8712-afadd51eda3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.522310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f174c-c252-41aa-8712-afadd51eda3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.524185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f174c-c252-41aa-8712-afadd51eda3f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.552439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1200/2398] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:21 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:21.556493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3b833-84ff-4934-98bf-244ad5426b5f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.567123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3b833-84ff-4934-98bf-244ad5426b5f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.622664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd7da4-98bf-48b7-81a9-e8b514732eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.623304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd7da4-98bf-48b7-81a9-e8b514732eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.623584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd7da4-98bf-48b7-81a9-e8b514732eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.623855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd7da4-98bf-48b7-81a9-e8b514732eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.624479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd7da4-98bf-48b7-81a9-e8b514732eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.626399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd7da4-98bf-48b7-81a9-e8b514732eb2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.647007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1199/2399] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:21 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:21.651432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f174c-c252-41aa-8712-afadd51eda3f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.662539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f174c-c252-41aa-8712-afadd51eda3f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.668359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533bd3a-7677-4c67-aacd-cbb7d49b8e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.669070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533bd3a-7677-4c67-aacd-cbb7d49b8e55 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.669430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533bd3a-7677-4c67-aacd-cbb7d49b8e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.669901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533bd3a-7677-4c67-aacd-cbb7d49b8e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.670598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533bd3a-7677-4c67-aacd-cbb7d49b8e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.672340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533bd3a-7677-4c67-aacd-cbb7d49b8e55 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.684263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1201/2400] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:34:21.688528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd7da4-98bf-48b7-81a9-e8b514732eb2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.699451 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd7da4-98bf-48b7-81a9-e8b514732eb2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.704290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f617de7-2518-4929-8b83-7958db64d765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.704991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f617de7-2518-4929-8b83-7958db64d765 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.705283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f617de7-2518-4929-8b83-7958db64d765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.705563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f617de7-2518-4929-8b83-7958db64d765 admin admin] PATH_INFO: `/v3/projects/c4d45b2ee1b2431190afdf4e62fc6289/users/bc11e2c131214a7395d23cfb1e643c47/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.721762 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f617de7-2518-4929-8b83-7958db64d765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d45b2ee1b2431190afdf4e62fc6289, user_id=bc11e2c131214a7395d23cfb1e643c47, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.724421 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f617de7-2518-4929-8b83-7958db64d765 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.737865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1200/2401] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:21 2022] PUT /identity/v3/projects/c4d45b2ee1b2431190afdf4e62fc6289/users/bc11e2c131214a7395d23cfb1e643c47/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:21.741867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533bd3a-7677-4c67-aacd-cbb7d49b8e55 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.751834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533bd3a-7677-4c67-aacd-cbb7d49b8e55 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.810500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce47b6-fa5a-4099-a4b7-a54147a27bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.811057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce47b6-fa5a-4099-a4b7-a54147a27bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.811394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce47b6-fa5a-4099-a4b7-a54147a27bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.811673 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce47b6-fa5a-4099-a4b7-a54147a27bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.812348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce47b6-fa5a-4099-a4b7-a54147a27bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.814222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce47b6-fa5a-4099-a4b7-a54147a27bc3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.829016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1202/2402] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:34:21.830246 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f617de7-2518-4929-8b83-7958db64d765 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.841824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f617de7-2518-4929-8b83-7958db64d765 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:21.847232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3863350-7b8f-41fa-bddf-e7588dd3ac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:21.847904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3863350-7b8f-41fa-bddf-e7588dd3ac06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.848189 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3863350-7b8f-41fa-bddf-e7588dd3ac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.848468 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3863350-7b8f-41fa-bddf-e7588dd3ac06 admin admin] PATH_INFO: `/v3/projects/c4d45b2ee1b2431190afdf4e62fc6289/users/bc11e2c131214a7395d23cfb1e643c47/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.850416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3863350-7b8f-41fa-bddf-e7588dd3ac06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d45b2ee1b2431190afdf4e62fc6289, user_id=bc11e2c131214a7395d23cfb1e643c47, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:21.853110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3863350-7b8f-41fa-bddf-e7588dd3ac06 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:21.859956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:21.865650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1201/2403] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:21 2022] PUT /identity/v3/projects/c4d45b2ee1b2431190afdf4e62fc6289/users/bc11e2c131214a7395d23cfb1e643c47/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:21.876042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb8f93-17c5-4d2d-b796-b04dafab980d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:21.876359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb8f93-17c5-4d2d-b796-b04dafab980d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:21.876610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb8f93-17c5-4d2d-b796-b04dafab980d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:21.907555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e2cb8f93-17c5-4d2d-b796-b04dafab980d None None] MFA Rules not processed for user `bc11e2c131214a7395d23cfb1e643c47`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:21.908776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e2cb8f93-17c5-4d2d-b796-b04dafab980d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:21.983491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1203/2404] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:21.993265 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3863350-7b8f-41fa-bddf-e7588dd3ac06 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:21.995348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e3863350-7b8f-41fa-bddf-e7588dd3ac06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:22.061559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3863350-7b8f-41fa-bddf-e7588dd3ac06 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:22.065157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e09a-38df-4934-8f48-91eb465eec65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:22.065769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e09a-38df-4934-8f48-91eb465eec65 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:22.066079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e09a-38df-4934-8f48-91eb465eec65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:22.066806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e09a-38df-4934-8f48-91eb465eec65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:22.141412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30e09a-38df-4934-8f48-91eb465eec65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:22.142870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30e09a-38df-4934-8f48-91eb465eec65 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:22.155848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1202/2405] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:23.743353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb8f93-17c5-4d2d-b796-b04dafab980d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:23.744822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e2cb8f93-17c5-4d2d-b796-b04dafab980d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:23.756773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb8f93-17c5-4d2d-b796-b04dafab980d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:23.760463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51d6cc-a0eb-4e1e-96fb-3e6a48a38124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:23.761064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51d6cc-a0eb-4e1e-96fb-3e6a48a38124 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:23.761344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51d6cc-a0eb-4e1e-96fb-3e6a48a38124 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:23.761617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51d6cc-a0eb-4e1e-96fb-3e6a48a38124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:23.782803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:23.821897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51d6cc-a0eb-4e1e-96fb-3e6a48a38124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:23.823356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51d6cc-a0eb-4e1e-96fb-3e6a48a38124 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:23.832674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1204/2406] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:24.353231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e09a-38df-4934-8f48-91eb465eec65 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:24.362059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e09a-38df-4934-8f48-91eb465eec65 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:24.365614 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecb3db-ae8c-4057-8b4b-5baf35412bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:24.366236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecb3db-ae8c-4057-8b4b-5baf35412bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:24.366516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecb3db-ae8c-4057-8b4b-5baf35412bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:24.366781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecb3db-ae8c-4057-8b4b-5baf35412bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:24.376315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecb3db-ae8c-4057-8b4b-5baf35412bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:24.377740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecb3db-ae8c-4057-8b4b-5baf35412bb8 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:24.388100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1203/2407] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:26.940079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51d6cc-a0eb-4e1e-96fb-3e6a48a38124 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:26.941678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7f51d6cc-a0eb-4e1e-96fb-3e6a48a38124 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:27.004380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51d6cc-a0eb-4e1e-96fb-3e6a48a38124 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:27.008185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb42f6-5f6a-40af-84cb-aef5dd75aef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:27.009580 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb42f6-5f6a-40af-84cb-aef5dd75aef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:27.010255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb42f6-5f6a-40af-84cb-aef5dd75aef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:27.010866 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb42f6-5f6a-40af-84cb-aef5dd75aef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:27.012884 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0ddb42f6-5f6a-40af-84cb-aef5dd75aef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:27.025057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb42f6-5f6a-40af-84cb-aef5dd75aef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:27.026784 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb42f6-5f6a-40af-84cb-aef5dd75aef5 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:27.037397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1205/2408] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:28.466611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecb3db-ae8c-4057-8b4b-5baf35412bb8 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:28.530938 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecb3db-ae8c-4057-8b4b-5baf35412bb8 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:28.535785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:28.536416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:28.536701 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:28.536981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] PATH_INFO: `/v3/users/2fa5b99453054be096f649fa18c029aa` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:28.539211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fa5b99453054be096f649fa18c029aa)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:28.540277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:28.603014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:28.605690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fa5b99453054be096f649fa18c029aa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:28.615813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fa5b99453054be096f649fa18c029aa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:28.617151 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fa5b99453054be096f649fa18c029aa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:28.617610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fa5b99453054be096f649fa18c029aa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:28.627879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fa5b99453054be096f649fa18c029aa'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:28.650544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1204/2409] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:28 2022] DELETE /identity/v3/users/2fa5b99453054be096f649fa18c029aa => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:28.654563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d64d23-369e-4ab7-8074-5ef36c83aa86 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:28.655056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d64d23-369e-4ab7-8074-5ef36c83aa86 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:28.655234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d64d23-369e-4ab7-8074-5ef36c83aa86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:28.674827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-33d64d23-369e-4ab7-8074-5ef36c83aa86 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:28.676122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-33d64d23-369e-4ab7-8074-5ef36c83aa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:28.743319 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1206/2410] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:28.755033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:28.756647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:28.824260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f54d1-7f6a-4de3-9e10-919a77d0df37 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:28.829064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8624f-4dcf-4b9e-b280-7c93bcdb84b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:28.829685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8624f-4dcf-4b9e-b280-7c93bcdb84b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:28.830000 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8624f-4dcf-4b9e-b280-7c93bcdb84b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:28.830277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8624f-4dcf-4b9e-b280-7c93bcdb84b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:28.846816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8624f-4dcf-4b9e-b280-7c93bcdb84b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:28.850499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8624f-4dcf-4b9e-b280-7c93bcdb84b1 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:28.861442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1205/2411] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:29.269060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d64d23-369e-4ab7-8074-5ef36c83aa86 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:29.271015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-33d64d23-369e-4ab7-8074-5ef36c83aa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:29.289812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d64d23-369e-4ab7-8074-5ef36c83aa86 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:29.297797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:29.299594 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:29.300039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:29.300476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] PATH_INFO: `/v3/projects/ab07c232383e444a9c75ab425c6a6fb3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:29.301677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab07c232383e444a9c75ab425c6a6fb3)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:29.303786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:29.338560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f5677-4e98-4fab-b47e-2bdcc3d3f56f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:29.338849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f5677-4e98-4fab-b47e-2bdcc3d3f56f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:29.339376 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f5677-4e98-4fab-b47e-2bdcc3d3f56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:29.362138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ca7f5677-4e98-4fab-b47e-2bdcc3d3f56f None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:29.363356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ca7f5677-4e98-4fab-b47e-2bdcc3d3f56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:29.376464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab07c232383e444a9c75ab425c6a6fb3', 'request_id': 'req-942d3a27-d01d-4b78-9586-b4a066a75a10'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.383109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab07c232383e444a9c75ab425c6a6fb3', 'request_id': 'req-942d3a27-d01d-4b78-9586-b4a066a75a10'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.386039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:29.395468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab07c232383e444a9c75ab425c6a6fb3', 'request_id': 'req-942d3a27-d01d-4b78-9586-b4a066a75a10'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.413492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] The token cache is being invalidate because project ab07c232383e444a9c75ab425c6a6fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:29.414241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.418195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.420174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1207/2412] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:29 2022] DELETE /identity/v3/projects/ab07c232383e444a9c75ab425c6a6fb3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:29.424892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:29.426754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:29.436658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1206/2413] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Sep 21 05:34:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:29.451008 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f5677-4e98-4fab-b47e-2bdcc3d3f56f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:29.452562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ca7f5677-4e98-4fab-b47e-2bdcc3d3f56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:29.517394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d3a27-d01d-4b78-9586-b4a066a75a10 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:29.522246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:29.522877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:29.523154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:29.523596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] PATH_INFO: `/v3/users/4036c8b2d4054027a9817fc06757d807` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:29.524876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=4036c8b2d4054027a9817fc06757d807)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:29.526828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:29.542349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f5677-4e98-4fab-b47e-2bdcc3d3f56f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:29.547493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272ccc5-3876-4b08-b7bf-2777880d97c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:29.548106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272ccc5-3876-4b08-b7bf-2777880d97c1 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:29.548388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272ccc5-3876-4b08-b7bf-2777880d97c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:29.548664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272ccc5-3876-4b08-b7bf-2777880d97c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:29.558645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272ccc5-3876-4b08-b7bf-2777880d97c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:29.561198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272ccc5-3876-4b08-b7bf-2777880d97c1 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:29.565455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:29.571149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1207/2414] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:34:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:34:29.584882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4036c8b2d4054027a9817fc06757d807'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.585676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4036c8b2d4054027a9817fc06757d807'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.595741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4036c8b2d4054027a9817fc06757d807'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.597959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4036c8b2d4054027a9817fc06757d807'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.609419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4036c8b2d4054027a9817fc06757d807'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:29.641135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1208/2415] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:29 2022] DELETE /identity/v3/users/4036c8b2d4054027a9817fc06757d807 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:30.120741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272ccc5-3876-4b08-b7bf-2777880d97c1 service swift] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:30.133357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272ccc5-3876-4b08-b7bf-2777880d97c1 service swift] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:30.212894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:30.213698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:30.214141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:30.214557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] PATH_INFO: `/v3/projects/944a1bc5b12a475d81ba90b4df8e7c83` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:30.216143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=944a1bc5b12a475d81ba90b4df8e7c83)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:30.218208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:30.265751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944a1bc5b12a475d81ba90b4df8e7c83', 'request_id': 'req-babf0c8a-8c84-426e-84fa-af3dc8a679e5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.272236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944a1bc5b12a475d81ba90b4df8e7c83', 'request_id': 'req-babf0c8a-8c84-426e-84fa-af3dc8a679e5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.278430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944a1bc5b12a475d81ba90b4df8e7c83', 'request_id': 'req-babf0c8a-8c84-426e-84fa-af3dc8a679e5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.290567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] The token cache is being invalidate because project 944a1bc5b12a475d81ba90b4df8e7c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:30.290865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.291523 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.293110 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1208/2416] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:30 2022] DELETE /identity/v3/projects/944a1bc5b12a475d81ba90b4df8e7c83 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:30.297156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:30.298700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:30.312517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757a5bb-231c-463e-b3e0-cdb7f5b294ec admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:30.394746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:30.395327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:30.395614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:30.395890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] PATH_INFO: `/v3/users/bc11e2c131214a7395d23cfb1e643c47` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:30.397028 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc11e2c131214a7395d23cfb1e643c47)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:30.399087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:30.411115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:30.457645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc11e2c131214a7395d23cfb1e643c47'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.458391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc11e2c131214a7395d23cfb1e643c47'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.468045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc11e2c131214a7395d23cfb1e643c47'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.468776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc11e2c131214a7395d23cfb1e643c47'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.478180 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-19ce1858-69ea-4eb9-a787-965381ee800c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc11e2c131214a7395d23cfb1e643c47'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:30.500192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1209/2417] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:30 2022] DELETE /identity/v3/users/bc11e2c131214a7395d23cfb1e643c47 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:30.909660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:30.911956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:30.933274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf0c8a-8c84-426e-84fa-af3dc8a679e5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:31.029718 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:31.030408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.034518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.034813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] PATH_INFO: `/v3/projects/c4d45b2ee1b2431190afdf4e62fc6289` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.036095 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d45b2ee1b2431190afdf4e62fc6289)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:31.038415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:31.080768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:31.113963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d45b2ee1b2431190afdf4e62fc6289', 'request_id': 'req-19226e65-52ec-41ac-a91c-27d5d07b2eba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:31.116254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d45b2ee1b2431190afdf4e62fc6289', 'request_id': 'req-19226e65-52ec-41ac-a91c-27d5d07b2eba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:31.132373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d45b2ee1b2431190afdf4e62fc6289', 'request_id': 'req-19226e65-52ec-41ac-a91c-27d5d07b2eba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:31.151597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] The token cache is being invalidate because project c4d45b2ee1b2431190afdf4e62fc6289 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:31.152744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:31.153449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19226e65-52ec-41ac-a91c-27d5d07b2eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:31.155359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1209/2418] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:30 2022] DELETE /identity/v3/projects/c4d45b2ee1b2431190afdf4e62fc6289 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:31.161847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868639e-a379-4aa6-a31b-14c2895b0785 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.162187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868639e-a379-4aa6-a31b-14c2895b0785 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.162538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868639e-a379-4aa6-a31b-14c2895b0785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.187165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4868639e-a379-4aa6-a31b-14c2895b0785 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:31.188700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4868639e-a379-4aa6-a31b-14c2895b0785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:31.286148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1210/2419] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:31.301400 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205480-a7a5-424a-b3fa-4583121dd6ce None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.301821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205480-a7a5-424a-b3fa-4583121dd6ce None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.302283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205480-a7a5-424a-b3fa-4583121dd6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.325331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ac205480-a7a5-424a-b3fa-4583121dd6ce None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:31.326782 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ac205480-a7a5-424a-b3fa-4583121dd6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:31.337975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1210/2420] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:31.341647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868639e-a379-4aa6-a31b-14c2895b0785 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:31.353979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868639e-a379-4aa6-a31b-14c2895b0785 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:31.359047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d6ce6-c4e2-4d61-ab56-a2018fc71220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:31.359810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d6ce6-c4e2-4d61-ab56-a2018fc71220 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.360195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d6ce6-c4e2-4d61-ab56-a2018fc71220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.360567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d6ce6-c4e2-4d61-ab56-a2018fc71220 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.361621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d6ce6-c4e2-4d61-ab56-a2018fc71220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:34:31.362069 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d6ce6-c4e2-4d61-ab56-a2018fc71220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:31.363824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d6ce6-c4e2-4d61-ab56-a2018fc71220 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:31.386994 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1211/2421] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:34:31.388015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac205480-a7a5-424a-b3fa-4583121dd6ce None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:31.406922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac205480-a7a5-424a-b3fa-4583121dd6ce None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:31.411824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51172e25-fb56-4080-80c6-dda6a2d269d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:31.412621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51172e25-fb56-4080-80c6-dda6a2d269d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.412741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51172e25-fb56-4080-80c6-dda6a2d269d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.413004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51172e25-fb56-4080-80c6-dda6a2d269d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.413646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51172e25-fb56-4080-80c6-dda6a2d269d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:31.416241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51172e25-fb56-4080-80c6-dda6a2d269d8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:31.461553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1211/2422] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:31 2022] POST /identity/v3/projects => generated 369 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:31.466531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d6ce6-c4e2-4d61-ab56-a2018fc71220 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:31.477355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d6ce6-c4e2-4d61-ab56-a2018fc71220 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:31.560271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afecaf-3f0f-4904-bd2c-bcf93ca5ba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:31.561231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afecaf-3f0f-4904-bd2c-bcf93ca5ba59 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.561680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afecaf-3f0f-4904-bd2c-bcf93ca5ba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.562267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afecaf-3f0f-4904-bd2c-bcf93ca5ba59 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.562899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afecaf-3f0f-4904-bd2c-bcf93ca5ba59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:31.564822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afecaf-3f0f-4904-bd2c-bcf93ca5ba59 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:31.593426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1212/2423] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:31 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:31.597774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51172e25-fb56-4080-80c6-dda6a2d269d8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:31.616562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51172e25-fb56-4080-80c6-dda6a2d269d8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:31.625221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9fb7a-035f-4f30-8ae5-429640b8b42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:31.625848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9fb7a-035f-4f30-8ae5-429640b8b42e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.626186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9fb7a-035f-4f30-8ae5-429640b8b42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.626464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9fb7a-035f-4f30-8ae5-429640b8b42e admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.627113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9fb7a-035f-4f30-8ae5-429640b8b42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:31.628883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9fb7a-035f-4f30-8ae5-429640b8b42e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:31.641391 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1212/2424] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:31.645760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afecaf-3f0f-4904-bd2c-bcf93ca5ba59 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:31.653592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:31.657442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afecaf-3f0f-4904-bd2c-bcf93ca5ba59 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:31.663343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9d62f-fc60-4f3a-a944-ba6e5f401e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:31.664119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d62f-fc60-4f3a-a944-ba6e5f401e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.664531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d62f-fc60-4f3a-a944-ba6e5f401e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.664900 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d62f-fc60-4f3a-a944-ba6e5f401e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.665741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9d62f-fc60-4f3a-a944-ba6e5f401e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:31.667370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9d62f-fc60-4f3a-a944-ba6e5f401e0a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:31.678151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1213/2425] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:34:31.682637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9fb7a-035f-4f30-8ae5-429640b8b42e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:31.693973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9fb7a-035f-4f30-8ae5-429640b8b42e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:31.699557 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dfd62-f135-4e66-bddd-0d54231a0b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:31.700377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dfd62-f135-4e66-bddd-0d54231a0b60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.700723 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dfd62-f135-4e66-bddd-0d54231a0b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.700947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dfd62-f135-4e66-bddd-0d54231a0b60 admin admin] PATH_INFO: `/v3/projects/b8710b7a992a463db3f8d1e78e784b77/users/55a5e9bc6c5347b9988851a26aa09813/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.717312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dfd62-f135-4e66-bddd-0d54231a0b60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8710b7a992a463db3f8d1e78e784b77, user_id=55a5e9bc6c5347b9988851a26aa09813, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:31.720435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dfd62-f135-4e66-bddd-0d54231a0b60 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:31.734742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1213/2426] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:31 2022] PUT /identity/v3/projects/b8710b7a992a463db3f8d1e78e784b77/users/55a5e9bc6c5347b9988851a26aa09813/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:31.748035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d15e8-1215-4657-bcac-ba6a4131c03c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.748403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d15e8-1215-4657-bcac-ba6a4131c03c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.748739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d15e8-1215-4657-bcac-ba6a4131c03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:31.784304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a78d15e8-1215-4657-bcac-ba6a4131c03c None None] MFA Rules not processed for user `55a5e9bc6c5347b9988851a26aa09813`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:31.785614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a78d15e8-1215-4657-bcac-ba6a4131c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:31.906983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1214/2427] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:31.917116 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dfd62-f135-4e66-bddd-0d54231a0b60 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:31.919532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b81dfd62-f135-4e66-bddd-0d54231a0b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:31.990697 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dfd62-f135-4e66-bddd-0d54231a0b60 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:31.994991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b23ca-682e-451b-bfb0-01339e4a1433 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5305f8a5d2b540de91d3caedad454bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:31.995639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b23ca-682e-451b-bfb0-01339e4a1433 service glance] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:31.995921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b23ca-682e-451b-bfb0-01339e4a1433 service glance] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:31.996195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b23ca-682e-451b-bfb0-01339e4a1433 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:32.005087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b23ca-682e-451b-bfb0-01339e4a1433 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:32.006559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b23ca-682e-451b-bfb0-01339e4a1433 service glance] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:32.016789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1214/2428] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:33.091476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d15e8-1215-4657-bcac-ba6a4131c03c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:33.093000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a78d15e8-1215-4657-bcac-ba6a4131c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:33.156337 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d15e8-1215-4657-bcac-ba6a4131c03c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:33.160130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1993a44-8932-4d87-8a52-3b0b28140b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:33.160742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1993a44-8932-4d87-8a52-3b0b28140b95 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:33.161049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1993a44-8932-4d87-8a52-3b0b28140b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:33.161356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1993a44-8932-4d87-8a52-3b0b28140b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:33.163162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d1993a44-8932-4d87-8a52-3b0b28140b95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:33.238383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1993a44-8932-4d87-8a52-3b0b28140b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:33.239901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1993a44-8932-4d87-8a52-3b0b28140b95 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:33.251274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1215/2429] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:37.489664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b23ca-682e-451b-bfb0-01339e4a1433 service glance] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:37.489664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-723b23ca-682e-451b-bfb0-01339e4a1433 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:37.506285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b23ca-682e-451b-bfb0-01339e4a1433 service glance] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:37.514852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:37.515374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:37.515645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:37.515936 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] PATH_INFO: `/v3/users/9fb2f0174dd04c0fb2aa0ac142ffce95` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:37.517135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb2f0174dd04c0fb2aa0ac142ffce95)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:37.519128 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:37.623069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb2f0174dd04c0fb2aa0ac142ffce95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:37.627493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:37.640282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb2f0174dd04c0fb2aa0ac142ffce95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:37.640912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb2f0174dd04c0fb2aa0ac142ffce95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:37.641595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb2f0174dd04c0fb2aa0ac142ffce95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:37.662667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dadbb4c1-92d6-4619-9f27-601b985fcb97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb2f0174dd04c0fb2aa0ac142ffce95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:37.688830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1215/2430] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:37 2022] DELETE /identity/v3/users/9fb2f0174dd04c0fb2aa0ac142ffce95 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:37.916327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1993a44-8932-4d87-8a52-3b0b28140b95 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:37.937333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1993a44-8932-4d87-8a52-3b0b28140b95 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:38.006136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:38.041968 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:38.042630 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.042916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.043191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] PATH_INFO: `/v3/projects/58c4b56493c94c2fb8e5e595cef89a55` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.044365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c4b56493c94c2fb8e5e595cef89a55)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:38.046415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:38.106034 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c4b56493c94c2fb8e5e595cef89a55', 'request_id': 'req-6065d4e4-8683-41ca-a799-02d895ff766f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:38.112106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c4b56493c94c2fb8e5e595cef89a55', 'request_id': 'req-6065d4e4-8683-41ca-a799-02d895ff766f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:38.121973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c4b56493c94c2fb8e5e595cef89a55', 'request_id': 'req-6065d4e4-8683-41ca-a799-02d895ff766f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:38.134469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] The token cache is being invalidate because project 58c4b56493c94c2fb8e5e595cef89a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:38.134794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:38.135406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6065d4e4-8683-41ca-a799-02d895ff766f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:38.137031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1216/2431] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:37 2022] DELETE /identity/v3/projects/58c4b56493c94c2fb8e5e595cef89a55 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:38.142671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8f099-211a-473f-9b5b-5c83732f1a90 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.142995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8f099-211a-473f-9b5b-5c83732f1a90 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.143435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8f099-211a-473f-9b5b-5c83732f1a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.166035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f4b8f099-211a-473f-9b5b-5c83732f1a90 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:38.167292 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f4b8f099-211a-473f-9b5b-5c83732f1a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:38.234355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1216/2432] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:38.243367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f2e3e-0824-4bd5-8e1e-977a0d6da5bd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.243573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f2e3e-0824-4bd5-8e1e-977a0d6da5bd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.243845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f2e3e-0824-4bd5-8e1e-977a0d6da5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.263965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f47f2e3e-0824-4bd5-8e1e-977a0d6da5bd None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:38.265233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f47f2e3e-0824-4bd5-8e1e-977a0d6da5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:38.273100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1217/2433] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:38.278130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8f099-211a-473f-9b5b-5c83732f1a90 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:38.290078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8f099-211a-473f-9b5b-5c83732f1a90 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:38.297437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c564cb-1ea0-478f-a82e-4811d37b52a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:38.298731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c564cb-1ea0-478f-a82e-4811d37b52a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.299090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c564cb-1ea0-478f-a82e-4811d37b52a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.299437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c564cb-1ea0-478f-a82e-4811d37b52a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.300164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c564cb-1ea0-478f-a82e-4811d37b52a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:34:38.300541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c564cb-1ea0-478f-a82e-4811d37b52a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:38.302795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c564cb-1ea0-478f-a82e-4811d37b52a2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:38.317445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1217/2434] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:34:38.321973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f2e3e-0824-4bd5-8e1e-977a0d6da5bd None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:38.331810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f2e3e-0824-4bd5-8e1e-977a0d6da5bd None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:38.337489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136adc73-d689-43cd-ab21-378eb48bf5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:38.338149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136adc73-d689-43cd-ab21-378eb48bf5ed admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.338433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136adc73-d689-43cd-ab21-378eb48bf5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.338708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136adc73-d689-43cd-ab21-378eb48bf5ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.339406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136adc73-d689-43cd-ab21-378eb48bf5ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:38.341556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136adc73-d689-43cd-ab21-378eb48bf5ed admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:38.376264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1218/2435] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:38 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:38.380473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c564cb-1ea0-478f-a82e-4811d37b52a2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:38.392990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c564cb-1ea0-478f-a82e-4811d37b52a2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:38.453853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af292c77-d4f8-4328-8bad-f48c1d5b290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:38.454596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af292c77-d4f8-4328-8bad-f48c1d5b290c admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.454943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af292c77-d4f8-4328-8bad-f48c1d5b290c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.455225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af292c77-d4f8-4328-8bad-f48c1d5b290c admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.455864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af292c77-d4f8-4328-8bad-f48c1d5b290c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:38.457786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af292c77-d4f8-4328-8bad-f48c1d5b290c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:38.486754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1218/2436] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:38 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:38.491336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136adc73-d689-43cd-ab21-378eb48bf5ed admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:38.502534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136adc73-d689-43cd-ab21-378eb48bf5ed admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:38.507712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4903648-c551-4e1f-9566-62ff5292da7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:38.508841 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4903648-c551-4e1f-9566-62ff5292da7d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.509182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4903648-c551-4e1f-9566-62ff5292da7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.509465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4903648-c551-4e1f-9566-62ff5292da7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.510129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4903648-c551-4e1f-9566-62ff5292da7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:38.511740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4903648-c551-4e1f-9566-62ff5292da7d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:38.526211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1219/2437] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:38.527471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af292c77-d4f8-4328-8bad-f48c1d5b290c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:38.538165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af292c77-d4f8-4328-8bad-f48c1d5b290c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:38.543821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455256d4-0613-4884-8d36-716f6ff85fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:38.544583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455256d4-0613-4884-8d36-716f6ff85fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.547389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455256d4-0613-4884-8d36-716f6ff85fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.547775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455256d4-0613-4884-8d36-716f6ff85fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.548543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455256d4-0613-4884-8d36-716f6ff85fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:38.550308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455256d4-0613-4884-8d36-716f6ff85fcc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:38.567583 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4903648-c551-4e1f-9566-62ff5292da7d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:38.575608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1219/2438] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:38.581389 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4903648-c551-4e1f-9566-62ff5292da7d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:38.606676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56733365-9512-4dc6-bfa8-159bfaa34806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:38.607637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56733365-9512-4dc6-bfa8-159bfaa34806 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.608102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56733365-9512-4dc6-bfa8-159bfaa34806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.608556 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56733365-9512-4dc6-bfa8-159bfaa34806 admin admin] PATH_INFO: `/v3/projects/4711bb2e7ed649f19be64e6cb396080d/users/4d40ec7f214a44ec9f7e95b2e7c07843/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.632149 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56733365-9512-4dc6-bfa8-159bfaa34806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4711bb2e7ed649f19be64e6cb396080d, user_id=4d40ec7f214a44ec9f7e95b2e7c07843, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:38.636177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56733365-9512-4dc6-bfa8-159bfaa34806 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:38.681627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1220/2439] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:38 2022] PUT /identity/v3/projects/4711bb2e7ed649f19be64e6cb396080d/users/4d40ec7f214a44ec9f7e95b2e7c07843/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:38.692242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3cb29-5265-4cd5-adfc-4208fce2ca3d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.692570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3cb29-5265-4cd5-adfc-4208fce2ca3d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.692805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3cb29-5265-4cd5-adfc-4208fce2ca3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.742356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-84b3cb29-5265-4cd5-adfc-4208fce2ca3d None None] MFA Rules not processed for user `4d40ec7f214a44ec9f7e95b2e7c07843`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:38.744594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-84b3cb29-5265-4cd5-adfc-4208fce2ca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:38.782192 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:38.830880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1220/2440] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:38.835569 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56733365-9512-4dc6-bfa8-159bfaa34806 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:38.904771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56733365-9512-4dc6-bfa8-159bfaa34806 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:38.909744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5aa9c-e83e-4c69-b54c-86407a019d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:38.910395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5aa9c-e83e-4c69-b54c-86407a019d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:38.910674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5aa9c-e83e-4c69-b54c-86407a019d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:38.910946 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5aa9c-e83e-4c69-b54c-86407a019d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:38.911590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5aa9c-e83e-4c69-b54c-86407a019d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:38.913526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5aa9c-e83e-4c69-b54c-86407a019d0d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:38.918436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:38.945616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1221/2441] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:38 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:38.950471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3cb29-5265-4cd5-adfc-4208fce2ca3d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:38.961371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3cb29-5265-4cd5-adfc-4208fce2ca3d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:39.021783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc714c1-83d0-4ecd-acae-2da51cb6e593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:39.022434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc714c1-83d0-4ecd-acae-2da51cb6e593 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:39.022714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc714c1-83d0-4ecd-acae-2da51cb6e593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:39.022990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc714c1-83d0-4ecd-acae-2da51cb6e593 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:39.023629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc714c1-83d0-4ecd-acae-2da51cb6e593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:39.025668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc714c1-83d0-4ecd-acae-2da51cb6e593 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:39.057717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1221/2442] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:38 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:39.060221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5aa9c-e83e-4c69-b54c-86407a019d0d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:39.074897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5aa9c-e83e-4c69-b54c-86407a019d0d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:39.095863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4761c8a-c47a-46bc-aa95-06bf159a4fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:39.096495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4761c8a-c47a-46bc-aa95-06bf159a4fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:39.096776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4761c8a-c47a-46bc-aa95-06bf159a4fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:39.097048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4761c8a-c47a-46bc-aa95-06bf159a4fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:39.097699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4761c8a-c47a-46bc-aa95-06bf159a4fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:39.099334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4761c8a-c47a-46bc-aa95-06bf159a4fb8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:39.112479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1222/2443] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:39.118471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc714c1-83d0-4ecd-acae-2da51cb6e593 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:39.128996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc714c1-83d0-4ecd-acae-2da51cb6e593 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:39.134140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf4072-4cb4-431f-bef8-20e2a3d04185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:39.135444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf4072-4cb4-431f-bef8-20e2a3d04185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:39.135444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf4072-4cb4-431f-bef8-20e2a3d04185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:39.135444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf4072-4cb4-431f-bef8-20e2a3d04185 admin admin] PATH_INFO: `/v3/projects/364d24395cff4c27a7532c3b4a0833af/users/2afc00e5bb5245fc85b65b2738005352/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:39.158586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf4072-4cb4-431f-bef8-20e2a3d04185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=364d24395cff4c27a7532c3b4a0833af, user_id=2afc00e5bb5245fc85b65b2738005352, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:39.161580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf4072-4cb4-431f-bef8-20e2a3d04185 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:39.177049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1222/2444] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:39 2022] PUT /identity/v3/projects/364d24395cff4c27a7532c3b4a0833af/users/2afc00e5bb5245fc85b65b2738005352/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:39.181747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4761c8a-c47a-46bc-aa95-06bf159a4fb8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:39.196801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4761c8a-c47a-46bc-aa95-06bf159a4fb8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:39.263278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79d145-a8c0-44b0-95b5-dd74f5f55e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:39.264135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79d145-a8c0-44b0-95b5-dd74f5f55e9b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:39.264437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79d145-a8c0-44b0-95b5-dd74f5f55e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:39.264712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79d145-a8c0-44b0-95b5-dd74f5f55e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:39.265354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79d145-a8c0-44b0-95b5-dd74f5f55e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:39.266993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79d145-a8c0-44b0-95b5-dd74f5f55e9b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:39.277323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1223/2445] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:39.281713 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf4072-4cb4-431f-bef8-20e2a3d04185 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:39.293633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf4072-4cb4-431f-bef8-20e2a3d04185 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:39.298431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab788b-626a-468d-9582-876ec45b9fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:39.299123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab788b-626a-468d-9582-876ec45b9fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:39.299409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab788b-626a-468d-9582-876ec45b9fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:39.299690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab788b-626a-468d-9582-876ec45b9fdf admin admin] PATH_INFO: `/v3/projects/364d24395cff4c27a7532c3b4a0833af/users/2afc00e5bb5245fc85b65b2738005352/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:39.301615 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab788b-626a-468d-9582-876ec45b9fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=364d24395cff4c27a7532c3b4a0833af, user_id=2afc00e5bb5245fc85b65b2738005352, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:39.304375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab788b-626a-468d-9582-876ec45b9fdf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:39.318696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1223/2446] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:39 2022] PUT /identity/v3/projects/364d24395cff4c27a7532c3b4a0833af/users/2afc00e5bb5245fc85b65b2738005352/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:39.329893 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed468c-9060-41a3-b315-8ad42444f57a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:39.330218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed468c-9060-41a3-b315-8ad42444f57a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:39.330505 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed468c-9060-41a3-b315-8ad42444f57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:39.360033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d3ed468c-9060-41a3-b315-8ad42444f57a None None] MFA Rules not processed for user `2afc00e5bb5245fc85b65b2738005352`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:39.361533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d3ed468c-9060-41a3-b315-8ad42444f57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:39.433547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1224/2447] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:39.444763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab788b-626a-468d-9582-876ec45b9fdf admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:39.446482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b9ab788b-626a-468d-9582-876ec45b9fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:39.505736 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab788b-626a-468d-9582-876ec45b9fdf admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:39.509423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd0499-269b-4799-947e-d1eb20207af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:39.510055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd0499-269b-4799-947e-d1eb20207af4 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:39.510340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd0499-269b-4799-947e-d1eb20207af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:39.510621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd0499-269b-4799-947e-d1eb20207af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:39.520024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd0499-269b-4799-947e-d1eb20207af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:39.521478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd0499-269b-4799-947e-d1eb20207af4 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:39.532022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1224/2448] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:39.977133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed468c-9060-41a3-b315-8ad42444f57a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:40.035195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:40.042131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed468c-9060-41a3-b315-8ad42444f57a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:40.046927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:40.047551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:40.047851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:40.048123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] PATH_INFO: `/v3/users/4d40ec7f214a44ec9f7e95b2e7c07843` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:40.049223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d40ec7f214a44ec9f7e95b2e7c07843)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:40.051158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:40.098470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d40ec7f214a44ec9f7e95b2e7c07843'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.099130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d40ec7f214a44ec9f7e95b2e7c07843'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.107971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d40ec7f214a44ec9f7e95b2e7c07843'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.108942 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d40ec7f214a44ec9f7e95b2e7c07843'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.116415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d40ec7f214a44ec9f7e95b2e7c07843'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.141490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1225/2449] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:39 2022] DELETE /identity/v3/users/4d40ec7f214a44ec9f7e95b2e7c07843 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:40.146138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0153d-8df4-4474-8841-634e29cc94d6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:40.146430 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0153d-8df4-4474-8841-634e29cc94d6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:40.146773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0153d-8df4-4474-8841-634e29cc94d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:40.166787 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-48a0153d-8df4-4474-8841-634e29cc94d6 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:40.167987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48a0153d-8df4-4474-8841-634e29cc94d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:40.181460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:40.228424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1225/2450] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:40.238312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:40.239701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:40.299495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9807b24-720a-4698-b624-c7e36bcf096c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:40.303136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffad64-8045-4261-b11d-68f2da6af280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:40.303740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffad64-8045-4261-b11d-68f2da6af280 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:40.304015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffad64-8045-4261-b11d-68f2da6af280 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:40.304283 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffad64-8045-4261-b11d-68f2da6af280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:40.314186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffad64-8045-4261-b11d-68f2da6af280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:40.315724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffad64-8045-4261-b11d-68f2da6af280 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:40.326913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1226/2451] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:40.601704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0153d-8df4-4474-8841-634e29cc94d6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:40.603686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48a0153d-8df4-4474-8841-634e29cc94d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:40.616906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0153d-8df4-4474-8841-634e29cc94d6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:40.623032 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:40.623752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:40.624025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:40.624301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] PATH_INFO: `/v3/projects/4711bb2e7ed649f19be64e6cb396080d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:40.625455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4711bb2e7ed649f19be64e6cb396080d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:40.628289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:40.689867 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4711bb2e7ed649f19be64e6cb396080d', 'request_id': 'req-10e9755c-a912-4891-bed2-5133074fd65f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.698233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4711bb2e7ed649f19be64e6cb396080d', 'request_id': 'req-10e9755c-a912-4891-bed2-5133074fd65f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.704644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4711bb2e7ed649f19be64e6cb396080d', 'request_id': 'req-10e9755c-a912-4891-bed2-5133074fd65f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.717243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] The token cache is being invalidate because project 4711bb2e7ed649f19be64e6cb396080d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:40.717688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.718399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.722980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1226/2452] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:40 2022] DELETE /identity/v3/projects/4711bb2e7ed649f19be64e6cb396080d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:40.726309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffad64-8045-4261-b11d-68f2da6af280 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:40.736655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffad64-8045-4261-b11d-68f2da6af280 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:40.802781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:40.803416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:40.803699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:40.803976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] PATH_INFO: `/v3/users/2afc00e5bb5245fc85b65b2738005352` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:40.805102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2afc00e5bb5245fc85b65b2738005352)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:40.807739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:40.830639 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:40.860459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2afc00e5bb5245fc85b65b2738005352'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.861487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2afc00e5bb5245fc85b65b2738005352'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.870433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2afc00e5bb5245fc85b65b2738005352'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.871055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2afc00e5bb5245fc85b65b2738005352'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.878407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cf32c9da-cbc5-4c66-aeb8-169036f2e5e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2afc00e5bb5245fc85b65b2738005352'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:40.898324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1227/2453] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:40 2022] DELETE /identity/v3/users/2afc00e5bb5245fc85b65b2738005352 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:41.287927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:41.289406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:41.309630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9755c-a912-4891-bed2-5133074fd65f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:41.382198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:41.382832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.383107 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.383910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] PATH_INFO: `/v3/projects/364d24395cff4c27a7532c3b4a0833af` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.385015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=364d24395cff4c27a7532c3b4a0833af)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:41.386981 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:41.430236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:41.433171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364d24395cff4c27a7532c3b4a0833af', 'request_id': 'req-77aaf025-08af-4af0-b141-be7a41d64cd1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:41.449968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364d24395cff4c27a7532c3b4a0833af', 'request_id': 'req-77aaf025-08af-4af0-b141-be7a41d64cd1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:41.456857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364d24395cff4c27a7532c3b4a0833af', 'request_id': 'req-77aaf025-08af-4af0-b141-be7a41d64cd1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:41.470731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] The token cache is being invalidate because project 364d24395cff4c27a7532c3b4a0833af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:41.470949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:41.471607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77aaf025-08af-4af0-b141-be7a41d64cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:41.473220 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1227/2454] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:41 2022] DELETE /identity/v3/projects/364d24395cff4c27a7532c3b4a0833af => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:41.478423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffe036-3b49-41cf-93ef-1f1eee65bbe6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.478890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffe036-3b49-41cf-93ef-1f1eee65bbe6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.479075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffe036-3b49-41cf-93ef-1f1eee65bbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.503332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c3ffe036-3b49-41cf-93ef-1f1eee65bbe6 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:41.504562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c3ffe036-3b49-41cf-93ef-1f1eee65bbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:41.567794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1228/2455] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:41.576106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09023a6-3bcf-433b-822e-bdfc059f7400 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.576397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09023a6-3bcf-433b-822e-bdfc059f7400 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.576712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09023a6-3bcf-433b-822e-bdfc059f7400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.598090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a09023a6-3bcf-433b-822e-bdfc059f7400 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:41.599307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a09023a6-3bcf-433b-822e-bdfc059f7400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:41.607495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1228/2456] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:41.615451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffe036-3b49-41cf-93ef-1f1eee65bbe6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:41.626939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffe036-3b49-41cf-93ef-1f1eee65bbe6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:41.632195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dbe3f-88ad-4963-a571-5e05739aec92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:41.632896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dbe3f-88ad-4963-a571-5e05739aec92 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.633185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dbe3f-88ad-4963-a571-5e05739aec92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.634160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dbe3f-88ad-4963-a571-5e05739aec92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.634818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dbe3f-88ad-4963-a571-5e05739aec92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:34:41.635252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dbe3f-88ad-4963-a571-5e05739aec92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:41.636845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dbe3f-88ad-4963-a571-5e05739aec92 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:41.648690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1229/2457] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:34:41.653052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09023a6-3bcf-433b-822e-bdfc059f7400 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:41.662942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09023a6-3bcf-433b-822e-bdfc059f7400 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:41.668450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc351-c8c6-4c88-b4e9-e51847365b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:41.669101 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc351-c8c6-4c88-b4e9-e51847365b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.669384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc351-c8c6-4c88-b4e9-e51847365b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.669662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc351-c8c6-4c88-b4e9-e51847365b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.670362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cc351-c8c6-4c88-b4e9-e51847365b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:41.672276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cc351-c8c6-4c88-b4e9-e51847365b39 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:41.701077 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1229/2458] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:41 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:41.705118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dbe3f-88ad-4963-a571-5e05739aec92 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:41.715978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dbe3f-88ad-4963-a571-5e05739aec92 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:41.779804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50205df3-4cb2-453d-b9d2-0629cb336396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:41.780618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50205df3-4cb2-453d-b9d2-0629cb336396 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.781087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50205df3-4cb2-453d-b9d2-0629cb336396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.781365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50205df3-4cb2-453d-b9d2-0629cb336396 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.782048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50205df3-4cb2-453d-b9d2-0629cb336396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:41.783972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50205df3-4cb2-453d-b9d2-0629cb336396 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:41.806990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1230/2459] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:41 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:41.811073 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc351-c8c6-4c88-b4e9-e51847365b39 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:41.821894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc351-c8c6-4c88-b4e9-e51847365b39 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:41.826520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98197e5-d69b-4c97-bd30-192333830604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:41.827126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98197e5-d69b-4c97-bd30-192333830604 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.827406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98197e5-d69b-4c97-bd30-192333830604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.827680 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98197e5-d69b-4c97-bd30-192333830604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.828324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98197e5-d69b-4c97-bd30-192333830604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:41.829957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98197e5-d69b-4c97-bd30-192333830604 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:41.840773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1230/2460] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:41.844724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50205df3-4cb2-453d-b9d2-0629cb336396 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:41.854675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50205df3-4cb2-453d-b9d2-0629cb336396 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:41.859396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce081d-baaa-41b2-aa85-2a4bf3b88daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:41.860073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce081d-baaa-41b2-aa85-2a4bf3b88daf admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.860356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce081d-baaa-41b2-aa85-2a4bf3b88daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.860634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce081d-baaa-41b2-aa85-2a4bf3b88daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.861616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce081d-baaa-41b2-aa85-2a4bf3b88daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:41.863236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce081d-baaa-41b2-aa85-2a4bf3b88daf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:41.875162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1231/2461] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:41.879454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98197e5-d69b-4c97-bd30-192333830604 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:41.891624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98197e5-d69b-4c97-bd30-192333830604 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:41.898464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245ee97-75ee-4e0a-840f-06e9452aa39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:41.899239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245ee97-75ee-4e0a-840f-06e9452aa39d admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.899578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245ee97-75ee-4e0a-840f-06e9452aa39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.899947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245ee97-75ee-4e0a-840f-06e9452aa39d admin admin] PATH_INFO: `/v3/projects/2c290884b6f1455dae39dab4d02f6ec2/users/57307f488f454e3e868ce212bb7f28a5/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.915514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245ee97-75ee-4e0a-840f-06e9452aa39d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c290884b6f1455dae39dab4d02f6ec2, user_id=57307f488f454e3e868ce212bb7f28a5, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:41.918228 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245ee97-75ee-4e0a-840f-06e9452aa39d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:41.932424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1231/2462] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:41 2022] PUT /identity/v3/projects/2c290884b6f1455dae39dab4d02f6ec2/users/57307f488f454e3e868ce212bb7f28a5/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:41.948654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002e669-6835-40da-a7d7-a28afe9692e6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:41.948940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002e669-6835-40da-a7d7-a28afe9692e6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:41.949234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002e669-6835-40da-a7d7-a28afe9692e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:41.956576 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:41.980745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e002e669-6835-40da-a7d7-a28afe9692e6 None None] MFA Rules not processed for user `57307f488f454e3e868ce212bb7f28a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:41.982169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e002e669-6835-40da-a7d7-a28afe9692e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:42.067207 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1232/2463] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:42.071692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245ee97-75ee-4e0a-840f-06e9452aa39d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:42.134767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245ee97-75ee-4e0a-840f-06e9452aa39d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:42.139394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3a9f4-38ed-4884-9eb9-2c091df94e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:42.139997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3a9f4-38ed-4884-9eb9-2c091df94e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:42.140277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3a9f4-38ed-4884-9eb9-2c091df94e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:42.140551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3a9f4-38ed-4884-9eb9-2c091df94e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:42.141188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3a9f4-38ed-4884-9eb9-2c091df94e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:42.143120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3a9f4-38ed-4884-9eb9-2c091df94e1f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:42.169598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1232/2464] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:42 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:42.173786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002e669-6835-40da-a7d7-a28afe9692e6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:42.185891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002e669-6835-40da-a7d7-a28afe9692e6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:42.262376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660dc1b-3cf6-4a9b-9437-e58ef0569c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:42.263290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660dc1b-3cf6-4a9b-9437-e58ef0569c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:42.263804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660dc1b-3cf6-4a9b-9437-e58ef0569c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:42.264339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660dc1b-3cf6-4a9b-9437-e58ef0569c3f admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:42.265280 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660dc1b-3cf6-4a9b-9437-e58ef0569c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:42.267733 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660dc1b-3cf6-4a9b-9437-e58ef0569c3f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:42.313987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1233/2465] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:42 2022] POST /identity/v3/users => generated 300 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:42.318311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3a9f4-38ed-4884-9eb9-2c091df94e1f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:42.334694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3a9f4-38ed-4884-9eb9-2c091df94e1f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:42.341588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f0b31-4702-4d72-bd19-382501190878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:42.342460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f0b31-4702-4d72-bd19-382501190878 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:42.343340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f0b31-4702-4d72-bd19-382501190878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:42.343735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f0b31-4702-4d72-bd19-382501190878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:42.344411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f0b31-4702-4d72-bd19-382501190878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:42.346589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f0b31-4702-4d72-bd19-382501190878 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:42.359303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1233/2466] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:42.363641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660dc1b-3cf6-4a9b-9437-e58ef0569c3f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:42.374827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660dc1b-3cf6-4a9b-9437-e58ef0569c3f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:42.382321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d65690-6c2f-4800-a0e6-1e2bfce6a480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:42.383221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d65690-6c2f-4800-a0e6-1e2bfce6a480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:42.384186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d65690-6c2f-4800-a0e6-1e2bfce6a480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:42.384490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d65690-6c2f-4800-a0e6-1e2bfce6a480 admin admin] PATH_INFO: `/v3/projects/4d968b3730484cceabbcd33bb4f0075c/users/ed788030ecaf47b5aba2b77f63177e8a/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:42.402917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d65690-6c2f-4800-a0e6-1e2bfce6a480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d968b3730484cceabbcd33bb4f0075c, user_id=ed788030ecaf47b5aba2b77f63177e8a, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:42.406150 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d65690-6c2f-4800-a0e6-1e2bfce6a480 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:42.425666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1234/2467] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:42 2022] PUT /identity/v3/projects/4d968b3730484cceabbcd33bb4f0075c/users/ed788030ecaf47b5aba2b77f63177e8a/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:42.430875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f0b31-4702-4d72-bd19-382501190878 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:42.444727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f0b31-4702-4d72-bd19-382501190878 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:42.489162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:42.506232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01628be2-1c09-430a-b99d-a0339d5306d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:42.506773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01628be2-1c09-430a-b99d-a0339d5306d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:42.507083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01628be2-1c09-430a-b99d-a0339d5306d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:42.507357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01628be2-1c09-430a-b99d-a0339d5306d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:42.507999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01628be2-1c09-430a-b99d-a0339d5306d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:42.510200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01628be2-1c09-430a-b99d-a0339d5306d9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:42.520516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1234/2468] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:42.524760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d65690-6c2f-4800-a0e6-1e2bfce6a480 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:42.535762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d65690-6c2f-4800-a0e6-1e2bfce6a480 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:42.540668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38d3c5-049c-4ab5-9f1b-123b02e89b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:42.541379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38d3c5-049c-4ab5-9f1b-123b02e89b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:42.541680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38d3c5-049c-4ab5-9f1b-123b02e89b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:42.541979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38d3c5-049c-4ab5-9f1b-123b02e89b92 admin admin] PATH_INFO: `/v3/projects/4d968b3730484cceabbcd33bb4f0075c/users/ed788030ecaf47b5aba2b77f63177e8a/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:42.543860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38d3c5-049c-4ab5-9f1b-123b02e89b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d968b3730484cceabbcd33bb4f0075c, user_id=ed788030ecaf47b5aba2b77f63177e8a, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:42.546532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38d3c5-049c-4ab5-9f1b-123b02e89b92 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:42.558621 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1235/2469] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:42 2022] PUT /identity/v3/projects/4d968b3730484cceabbcd33bb4f0075c/users/ed788030ecaf47b5aba2b77f63177e8a/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:42.570344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877b028-b619-4eff-99bb-426d0b4de2de None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:42.570634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877b028-b619-4eff-99bb-426d0b4de2de None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:42.571481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877b028-b619-4eff-99bb-426d0b4de2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:42.601276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1877b028-b619-4eff-99bb-426d0b4de2de None None] MFA Rules not processed for user `ed788030ecaf47b5aba2b77f63177e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:42.602587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1877b028-b619-4eff-99bb-426d0b4de2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:42.684689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1235/2470] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:42.698411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38d3c5-049c-4ab5-9f1b-123b02e89b92 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:42.700196 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9a38d3c5-049c-4ab5-9f1b-123b02e89b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:42.767197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:42.781629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38d3c5-049c-4ab5-9f1b-123b02e89b92 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:42.785970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b243724-a337-46c4-843b-63fb7b7e1868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:42.786804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b243724-a337-46c4-843b-63fb7b7e1868 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:42.787087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b243724-a337-46c4-843b-63fb7b7e1868 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:42.787357 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b243724-a337-46c4-843b-63fb7b7e1868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:42.798007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b243724-a337-46c4-843b-63fb7b7e1868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:42.800044 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b243724-a337-46c4-843b-63fb7b7e1868 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:42.811741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1236/2471] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:42.844119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877b028-b619-4eff-99bb-426d0b4de2de None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:42.921916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877b028-b619-4eff-99bb-426d0b4de2de None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:42.926980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:42.927621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:42.927905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:42.928177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] PATH_INFO: `/v3/users/57307f488f454e3e868ce212bb7f28a5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:42.929306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57307f488f454e3e868ce212bb7f28a5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:42.931983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:42.992350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57307f488f454e3e868ce212bb7f28a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:43.003472 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57307f488f454e3e868ce212bb7f28a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:43.004358 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57307f488f454e3e868ce212bb7f28a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:43.004881 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57307f488f454e3e868ce212bb7f28a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:43.012990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57307f488f454e3e868ce212bb7f28a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:43.017988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:43.038317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1236/2472] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:42 2022] DELETE /identity/v3/users/57307f488f454e3e868ce212bb7f28a5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:43.043487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244373c-3e9c-474d-b888-af96283c677f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:43.043776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244373c-3e9c-474d-b888-af96283c677f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:43.044057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244373c-3e9c-474d-b888-af96283c677f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:43.064284 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f244373c-3e9c-474d-b888-af96283c677f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:43.065502 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f244373c-3e9c-474d-b888-af96283c677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:43.140429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1237/2473] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:43.154865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:43.159460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:43.304282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addad0ec-211b-4d6e-a515-c311fb841481 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:43.308386 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de2dfb-4d67-4456-ac16-f72030f3cc89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:43.309072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2dfb-4d67-4456-ac16-f72030f3cc89 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:43.309333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2dfb-4d67-4456-ac16-f72030f3cc89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:43.309825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2dfb-4d67-4456-ac16-f72030f3cc89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:43.323678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de2dfb-4d67-4456-ac16-f72030f3cc89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:43.325147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de2dfb-4d67-4456-ac16-f72030f3cc89 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:43.337849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1237/2474] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:43.885200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244373c-3e9c-474d-b888-af96283c677f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:43.887724 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f244373c-3e9c-474d-b888-af96283c677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:43.911980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244373c-3e9c-474d-b888-af96283c677f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:43.929281 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:43.930794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:43.931188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:43.931589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] PATH_INFO: `/v3/projects/2c290884b6f1455dae39dab4d02f6ec2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:43.932886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c290884b6f1455dae39dab4d02f6ec2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:43.935647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:44.013667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c290884b6f1455dae39dab4d02f6ec2', 'request_id': 'req-e9dedabb-e404-413e-adfc-ae28fe741bcf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.020469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c290884b6f1455dae39dab4d02f6ec2', 'request_id': 'req-e9dedabb-e404-413e-adfc-ae28fe741bcf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.030275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c290884b6f1455dae39dab4d02f6ec2', 'request_id': 'req-e9dedabb-e404-413e-adfc-ae28fe741bcf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.051858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] The token cache is being invalidate because project 2c290884b6f1455dae39dab4d02f6ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:44.052162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.052966 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.056101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1238/2475] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:43 2022] DELETE /identity/v3/projects/2c290884b6f1455dae39dab4d02f6ec2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:44.059802 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de2dfb-4d67-4456-ac16-f72030f3cc89 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:44.074388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de2dfb-4d67-4456-ac16-f72030f3cc89 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:44.153658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:44.154593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:44.154879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:44.155150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] PATH_INFO: `/v3/users/ed788030ecaf47b5aba2b77f63177e8a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:44.156274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed788030ecaf47b5aba2b77f63177e8a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:44.158206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:44.215666 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed788030ecaf47b5aba2b77f63177e8a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.223732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed788030ecaf47b5aba2b77f63177e8a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.224827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed788030ecaf47b5aba2b77f63177e8a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.225687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed788030ecaf47b5aba2b77f63177e8a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.235446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11644cc6-12fa-4ade-a2e1-d075c804af92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed788030ecaf47b5aba2b77f63177e8a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.257743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1238/2476] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:44 2022] DELETE /identity/v3/users/ed788030ecaf47b5aba2b77f63177e8a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:44.602983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:44.604771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:44.619896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dedabb-e404-413e-adfc-ae28fe741bcf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:44.681642 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:44.682335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:44.682611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:44.682879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] PATH_INFO: `/v3/projects/4d968b3730484cceabbcd33bb4f0075c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:44.684007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d968b3730484cceabbcd33bb4f0075c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:44.686064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:44.700813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:44.749102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b397f57-8498-4d5b-88a0-9adead61a7bf None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:44.749679 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b397f57-8498-4d5b-88a0-9adead61a7bf None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:44.750120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b397f57-8498-4d5b-88a0-9adead61a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:44.766689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:44.778413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3b397f57-8498-4d5b-88a0-9adead61a7bf None None] MFA Rules not processed for user `9d05e7bf6ff04193849987316fcedb81`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:44.780139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3b397f57-8498-4d5b-88a0-9adead61a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:44.783195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d968b3730484cceabbcd33bb4f0075c', 'request_id': 'req-a866d519-e990-4dbb-9914-20839c66a052'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.793419 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d968b3730484cceabbcd33bb4f0075c', 'request_id': 'req-a866d519-e990-4dbb-9914-20839c66a052'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.806348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d968b3730484cceabbcd33bb4f0075c', 'request_id': 'req-a866d519-e990-4dbb-9914-20839c66a052'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.822188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] The token cache is being invalidate because project 4d968b3730484cceabbcd33bb4f0075c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:44.822391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.823171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a866d519-e990-4dbb-9914-20839c66a052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:44.824758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1239/2477] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:44 2022] DELETE /identity/v3/projects/4d968b3730484cceabbcd33bb4f0075c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:44.829547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845f1ca-7ed1-4a57-91eb-caa0a6bbe655 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:44.829830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845f1ca-7ed1-4a57-91eb-caa0a6bbe655 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:44.830136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845f1ca-7ed1-4a57-91eb-caa0a6bbe655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:44.846866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1239/2478] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:44.850308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2845f1ca-7ed1-4a57-91eb-caa0a6bbe655 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:44.851859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2845f1ca-7ed1-4a57-91eb-caa0a6bbe655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:44.920184 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1240/2479] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:44.927922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e2918-ae88-4ec9-8df6-dfa6192afe5d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:44.928208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e2918-ae88-4ec9-8df6-dfa6192afe5d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:44.928480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e2918-ae88-4ec9-8df6-dfa6192afe5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:44.949924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-de1e2918-ae88-4ec9-8df6-dfa6192afe5d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:44.951188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-de1e2918-ae88-4ec9-8df6-dfa6192afe5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:44.960308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1240/2480] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:44.964646 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845f1ca-7ed1-4a57-91eb-caa0a6bbe655 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:44.974876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845f1ca-7ed1-4a57-91eb-caa0a6bbe655 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:44.979629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf54ad-5049-4c55-81d2-7fe86e39b9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:44.980241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf54ad-5049-4c55-81d2-7fe86e39b9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:44.980523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf54ad-5049-4c55-81d2-7fe86e39b9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:44.980797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf54ad-5049-4c55-81d2-7fe86e39b9b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:44.981443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf54ad-5049-4c55-81d2-7fe86e39b9b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:34:44.981749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf54ad-5049-4c55-81d2-7fe86e39b9b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:44.983550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf54ad-5049-4c55-81d2-7fe86e39b9b4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:44.996610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1241/2481] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:34:45.000822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e2918-ae88-4ec9-8df6-dfa6192afe5d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.011001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e2918-ae88-4ec9-8df6-dfa6192afe5d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.015632 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e38cd-ba62-4635-97eb-dc0bdde3f787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.016231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e38cd-ba62-4635-97eb-dc0bdde3f787 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.016507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e38cd-ba62-4635-97eb-dc0bdde3f787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.016778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e38cd-ba62-4635-97eb-dc0bdde3f787 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.017406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e38cd-ba62-4635-97eb-dc0bdde3f787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.019324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e38cd-ba62-4635-97eb-dc0bdde3f787 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.056295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1241/2482] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:44 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:45.057467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf54ad-5049-4c55-81d2-7fe86e39b9b4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.069107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf54ad-5049-4c55-81d2-7fe86e39b9b4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.128611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1a398-1225-4498-8626-5c040059e270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.129400 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1a398-1225-4498-8626-5c040059e270 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.129657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1a398-1225-4498-8626-5c040059e270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.130089 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1a398-1225-4498-8626-5c040059e270 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.133080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1a398-1225-4498-8626-5c040059e270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.133080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1a398-1225-4498-8626-5c040059e270 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.155981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1242/2483] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:45 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:45.159937 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e38cd-ba62-4635-97eb-dc0bdde3f787 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.173260 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e38cd-ba62-4635-97eb-dc0bdde3f787 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.178435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3e43e-8efb-4d23-9be5-56b32c481e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.179072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3e43e-8efb-4d23-9be5-56b32c481e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.179361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3e43e-8efb-4d23-9be5-56b32c481e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.179638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3e43e-8efb-4d23-9be5-56b32c481e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.181006 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3e43e-8efb-4d23-9be5-56b32c481e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.182690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3e43e-8efb-4d23-9be5-56b32c481e45 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.193970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1242/2484] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:45.197076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1a398-1225-4498-8626-5c040059e270 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.206854 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1a398-1225-4498-8626-5c040059e270 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.212066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215814e-1001-4534-96f5-575f462e5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.212672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215814e-1001-4534-96f5-575f462e5e9d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.212952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215814e-1001-4534-96f5-575f462e5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.213228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215814e-1001-4534-96f5-575f462e5e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.213868 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215814e-1001-4534-96f5-575f462e5e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.215501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215814e-1001-4534-96f5-575f462e5e9d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.227369 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1243/2485] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:34:45.231412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3e43e-8efb-4d23-9be5-56b32c481e45 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.242203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3e43e-8efb-4d23-9be5-56b32c481e45 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.248135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5bb0-4348-4025-b72a-ba634f2ebf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.248520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5bb0-4348-4025-b72a-ba634f2ebf1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.248810 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5bb0-4348-4025-b72a-ba634f2ebf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.249088 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5bb0-4348-4025-b72a-ba634f2ebf1a admin admin] PATH_INFO: `/v3/projects/84080466ccc94ee5ab95d7fab804734f/users/34fe049fe2034585ab3b56d19fb78fec/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.264645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c5bb0-4348-4025-b72a-ba634f2ebf1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=84080466ccc94ee5ab95d7fab804734f, user_id=34fe049fe2034585ab3b56d19fb78fec, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.267352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c5bb0-4348-4025-b72a-ba634f2ebf1a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.280501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1243/2486] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:45 2022] PUT /identity/v3/projects/84080466ccc94ee5ab95d7fab804734f/users/34fe049fe2034585ab3b56d19fb78fec/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:45.291506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df181f5d-f337-41c7-a736-daba604c4694 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.291789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df181f5d-f337-41c7-a736-daba604c4694 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.292061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df181f5d-f337-41c7-a736-daba604c4694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.306454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:45.323696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-df181f5d-f337-41c7-a736-daba604c4694 None None] MFA Rules not processed for user `34fe049fe2034585ab3b56d19fb78fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:45.324971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-df181f5d-f337-41c7-a736-daba604c4694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:45.395231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1244/2487] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:45.400282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5bb0-4348-4025-b72a-ba634f2ebf1a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.472345 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5bb0-4348-4025-b72a-ba634f2ebf1a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.477753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270a35-5da8-4052-94c3-ab2c67abddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.478568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270a35-5da8-4052-94c3-ab2c67abddf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.478702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270a35-5da8-4052-94c3-ab2c67abddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.478919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270a35-5da8-4052-94c3-ab2c67abddf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.479547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52270a35-5da8-4052-94c3-ab2c67abddf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.481422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52270a35-5da8-4052-94c3-ab2c67abddf1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.527080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1244/2488] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:45 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:45.528457 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df181f5d-f337-41c7-a736-daba604c4694 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.539311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df181f5d-f337-41c7-a736-daba604c4694 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.601397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b88c0-e4b2-4903-bf25-54cb15d1fd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.602627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b88c0-e4b2-4903-bf25-54cb15d1fd68 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.602909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b88c0-e4b2-4903-bf25-54cb15d1fd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.603298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b88c0-e4b2-4903-bf25-54cb15d1fd68 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.603951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b88c0-e4b2-4903-bf25-54cb15d1fd68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.606036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b88c0-e4b2-4903-bf25-54cb15d1fd68 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.632250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1245/2489] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:45 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:45.636247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270a35-5da8-4052-94c3-ab2c67abddf1 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.647904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270a35-5da8-4052-94c3-ab2c67abddf1 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.653395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ed0d1-c8ab-4b04-9b12-cc89247c0945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.654069 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ed0d1-c8ab-4b04-9b12-cc89247c0945 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.654354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ed0d1-c8ab-4b04-9b12-cc89247c0945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.654628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ed0d1-c8ab-4b04-9b12-cc89247c0945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.655273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ed0d1-c8ab-4b04-9b12-cc89247c0945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.656880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ed0d1-c8ab-4b04-9b12-cc89247c0945 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.670405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1245/2490] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:45.675370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b88c0-e4b2-4903-bf25-54cb15d1fd68 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.686605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b88c0-e4b2-4903-bf25-54cb15d1fd68 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.691584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33168030-3979-44fb-a80e-47b6466304a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.692259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33168030-3979-44fb-a80e-47b6466304a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.692543 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33168030-3979-44fb-a80e-47b6466304a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.693203 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33168030-3979-44fb-a80e-47b6466304a2 admin admin] PATH_INFO: `/v3/projects/249bb8dafd8041c08a4be7baf30f8330/users/c8147fb2e2ff4491b178f5301d3f9cfe/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.709517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33168030-3979-44fb-a80e-47b6466304a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249bb8dafd8041c08a4be7baf30f8330, user_id=c8147fb2e2ff4491b178f5301d3f9cfe, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.712822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33168030-3979-44fb-a80e-47b6466304a2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.728189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1246/2491] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:45 2022] PUT /identity/v3/projects/249bb8dafd8041c08a4be7baf30f8330/users/c8147fb2e2ff4491b178f5301d3f9cfe/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:45.732281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ed0d1-c8ab-4b04-9b12-cc89247c0945 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.744439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ed0d1-c8ab-4b04-9b12-cc89247c0945 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.776462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:45.856866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6dc93-84c6-41de-a7f1-461ddd3db82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.857515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6dc93-84c6-41de-a7f1-461ddd3db82c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.857800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6dc93-84c6-41de-a7f1-461ddd3db82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.858245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6dc93-84c6-41de-a7f1-461ddd3db82c admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.860509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6dc93-84c6-41de-a7f1-461ddd3db82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.862747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6dc93-84c6-41de-a7f1-461ddd3db82c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.879403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1246/2492] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:45.879777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33168030-3979-44fb-a80e-47b6466304a2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:45.893053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33168030-3979-44fb-a80e-47b6466304a2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:45.911697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0941745-5b08-4ee9-8054-17c5affa6685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:45.912531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0941745-5b08-4ee9-8054-17c5affa6685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.912921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0941745-5b08-4ee9-8054-17c5affa6685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.913258 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0941745-5b08-4ee9-8054-17c5affa6685 admin admin] PATH_INFO: `/v3/projects/249bb8dafd8041c08a4be7baf30f8330/users/c8147fb2e2ff4491b178f5301d3f9cfe/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.915710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0941745-5b08-4ee9-8054-17c5affa6685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249bb8dafd8041c08a4be7baf30f8330, user_id=c8147fb2e2ff4491b178f5301d3f9cfe, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:45.918528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0941745-5b08-4ee9-8054-17c5affa6685 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:45.932098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1247/2493] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:45 2022] PUT /identity/v3/projects/249bb8dafd8041c08a4be7baf30f8330/users/c8147fb2e2ff4491b178f5301d3f9cfe/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:45.943622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384804c6-75fd-464f-a41f-a9e73a906cce None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:45.944269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384804c6-75fd-464f-a41f-a9e73a906cce None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:45.944554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384804c6-75fd-464f-a41f-a9e73a906cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:45.978209 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-384804c6-75fd-464f-a41f-a9e73a906cce None None] MFA Rules not processed for user `c8147fb2e2ff4491b178f5301d3f9cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:45.979419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-384804c6-75fd-464f-a41f-a9e73a906cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:46.055897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1247/2494] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:46.067742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0941745-5b08-4ee9-8054-17c5affa6685 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:46.070183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f0941745-5b08-4ee9-8054-17c5affa6685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:46.137394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:46.141852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0941745-5b08-4ee9-8054-17c5affa6685 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:46.146486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbcca9-e7aa-4e5e-ae6b-3ea86f66bf5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:46.147166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbcca9-e7aa-4e5e-ae6b-3ea86f66bf5d service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:46.147572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbcca9-e7aa-4e5e-ae6b-3ea86f66bf5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:46.147892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbcca9-e7aa-4e5e-ae6b-3ea86f66bf5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:46.157302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbcca9-e7aa-4e5e-ae6b-3ea86f66bf5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:46.158869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbcca9-e7aa-4e5e-ae6b-3ea86f66bf5d service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:46.169123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1248/2495] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:46.236518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384804c6-75fd-464f-a41f-a9e73a906cce None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:46.298003 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384804c6-75fd-464f-a41f-a9e73a906cce None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:46.302641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:46.303274 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:46.303555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:46.303827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] PATH_INFO: `/v3/users/34fe049fe2034585ab3b56d19fb78fec` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:46.304948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] RBAC: Authorizing `identity:delete_user(user_id=34fe049fe2034585ab3b56d19fb78fec)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:46.307706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:46.355166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:46.357122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34fe049fe2034585ab3b56d19fb78fec'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:46.367082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34fe049fe2034585ab3b56d19fb78fec'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:46.368510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34fe049fe2034585ab3b56d19fb78fec'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:46.369112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34fe049fe2034585ab3b56d19fb78fec'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:46.376435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34fe049fe2034585ab3b56d19fb78fec'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:46.395973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1248/2496] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:46 2022] DELETE /identity/v3/users/34fe049fe2034585ab3b56d19fb78fec => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:46.400605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68058fd5-2cfb-4970-8514-6803a1d03db3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:46.400931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68058fd5-2cfb-4970-8514-6803a1d03db3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:46.401252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68058fd5-2cfb-4970-8514-6803a1d03db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:46.422488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-68058fd5-2cfb-4970-8514-6803a1d03db3 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:46.423731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-68058fd5-2cfb-4970-8514-6803a1d03db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:46.486616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1249/2497] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:46.497530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:46.499352 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:46.573524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224fa2d-80e4-442c-8535-c11c832294be admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:46.577883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb0c9d-edfe-4a7c-8ab3-00ef3f321ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:46.578651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb0c9d-edfe-4a7c-8ab3-00ef3f321ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:46.579026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb0c9d-edfe-4a7c-8ab3-00ef3f321ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:46.579343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb0c9d-edfe-4a7c-8ab3-00ef3f321ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:46.599978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb0c9d-edfe-4a7c-8ab3-00ef3f321ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:46.601448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb0c9d-edfe-4a7c-8ab3-00ef3f321ef9 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:46.614078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1249/2498] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:46.888735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68058fd5-2cfb-4970-8514-6803a1d03db3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:46.890210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-68058fd5-2cfb-4970-8514-6803a1d03db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:46.903085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68058fd5-2cfb-4970-8514-6803a1d03db3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:46.908229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:46.908938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:46.909295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:46.909622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] PATH_INFO: `/v3/projects/84080466ccc94ee5ab95d7fab804734f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:46.910776 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84080466ccc94ee5ab95d7fab804734f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:46.912975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:46.971185 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84080466ccc94ee5ab95d7fab804734f', 'request_id': 'req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:46.980051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84080466ccc94ee5ab95d7fab804734f', 'request_id': 'req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:46.989822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84080466ccc94ee5ab95d7fab804734f', 'request_id': 'req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.005074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] The token cache is being invalidate because project 84080466ccc94ee5ab95d7fab804734f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:47.005074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.005074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.005074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1250/2499] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:46 2022] DELETE /identity/v3/projects/84080466ccc94ee5ab95d7fab804734f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:47.008903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb0c9d-edfe-4a7c-8ab3-00ef3f321ef9 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:47.019222 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb0c9d-edfe-4a7c-8ab3-00ef3f321ef9 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:47.075988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:47.076607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:47.076884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:47.077156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] PATH_INFO: `/v3/users/c8147fb2e2ff4491b178f5301d3f9cfe` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:47.078288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8147fb2e2ff4491b178f5301d3f9cfe)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:47.080379 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:47.124665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8147fb2e2ff4491b178f5301d3f9cfe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.134914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8147fb2e2ff4491b178f5301d3f9cfe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.135551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8147fb2e2ff4491b178f5301d3f9cfe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.136223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8147fb2e2ff4491b178f5301d3f9cfe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.144262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25ac43e1-bb6f-4b0d-a52f-ecccd3da7b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8147fb2e2ff4491b178f5301d3f9cfe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.148551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:47.163694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1250/2500] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:47 2022] DELETE /identity/v3/users/c8147fb2e2ff4491b178f5301d3f9cfe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:47.591317 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:47.592906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:47.609539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b98d48-b425-45ce-b031-a3ec5bc11b0c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:47.677261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:47.678112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:47.678497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:47.678780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] PATH_INFO: `/v3/projects/249bb8dafd8041c08a4be7baf30f8330` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:47.680485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=249bb8dafd8041c08a4be7baf30f8330)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:47.682434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:47.700330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:47.731276 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '249bb8dafd8041c08a4be7baf30f8330', 'request_id': 'req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.737223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '249bb8dafd8041c08a4be7baf30f8330', 'request_id': 'req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.745810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '249bb8dafd8041c08a4be7baf30f8330', 'request_id': 'req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.757470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] The token cache is being invalidate because project 249bb8dafd8041c08a4be7baf30f8330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:47.757902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.759197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a7fa1694-3ce1-4d8a-bef1-2733dfc710d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:47.760682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1251/2501] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:47 2022] DELETE /identity/v3/projects/249bb8dafd8041c08a4be7baf30f8330 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:47.765565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc0090-2485-42d6-bc26-940754eadccb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:47.766076 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc0090-2485-42d6-bc26-940754eadccb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:47.766180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc0090-2485-42d6-bc26-940754eadccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:47.786502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-81cc0090-2485-42d6-bc26-940754eadccb None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:47.787860 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-81cc0090-2485-42d6-bc26-940754eadccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:47.847535 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1251/2502] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:47.855402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d4862-3478-4c85-8fb2-f5086e614c4f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:47.855687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d4862-3478-4c85-8fb2-f5086e614c4f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:47.855961 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d4862-3478-4c85-8fb2-f5086e614c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:47.876106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b82d4862-3478-4c85-8fb2-f5086e614c4f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:47.877321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b82d4862-3478-4c85-8fb2-f5086e614c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:47.885766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1252/2503] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:47.889888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc0090-2485-42d6-bc26-940754eadccb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:47.899625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc0090-2485-42d6-bc26-940754eadccb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:47.904150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f57e6-66ae-459e-8772-0fe1a44adf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:47.904746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f57e6-66ae-459e-8772-0fe1a44adf80 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:47.905029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f57e6-66ae-459e-8772-0fe1a44adf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:47.905306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f57e6-66ae-459e-8772-0fe1a44adf80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:47.905962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f57e6-66ae-459e-8772-0fe1a44adf80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:34:47.906308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f57e6-66ae-459e-8772-0fe1a44adf80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:47.907884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f57e6-66ae-459e-8772-0fe1a44adf80 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:47.919532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1252/2504] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:34:47.923857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d4862-3478-4c85-8fb2-f5086e614c4f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:47.933741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d4862-3478-4c85-8fb2-f5086e614c4f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:47.938446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb0b94-846d-4b3c-90b8-377880919c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:47.939047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb0b94-846d-4b3c-90b8-377880919c85 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:47.939328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb0b94-846d-4b3c-90b8-377880919c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:47.939599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb0b94-846d-4b3c-90b8-377880919c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:47.940226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb0b94-846d-4b3c-90b8-377880919c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:47.942152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb0b94-846d-4b3c-90b8-377880919c85 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:47.976805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1253/2505] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:47 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:47.977602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f57e6-66ae-459e-8772-0fe1a44adf80 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:47.980053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c83e3-a82c-46a2-8120-d14dcb40da19 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:47.980386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c83e3-a82c-46a2-8120-d14dcb40da19 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:47.980669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c83e3-a82c-46a2-8120-d14dcb40da19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:47.990177 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f57e6-66ae-459e-8772-0fe1a44adf80 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.000685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2d7c83e3-a82c-46a2-8120-d14dcb40da19 None None] MFA Rules not processed for user `d2297673f49b4ce8bf40ac5f750d989f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:48.001905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2d7c83e3-a82c-46a2-8120-d14dcb40da19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:48.075910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1254/2506] 10.222.0.18 () {58 vars in 967 bytes} [Wed Sep 21 05:34:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:48.080747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651fe40-b7cc-45cb-8784-250c3edbe0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.081428 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651fe40-b7cc-45cb-8784-250c3edbe0dd admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.081720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651fe40-b7cc-45cb-8784-250c3edbe0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.082047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651fe40-b7cc-45cb-8784-250c3edbe0dd admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.082716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651fe40-b7cc-45cb-8784-250c3edbe0dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.085858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651fe40-b7cc-45cb-8784-250c3edbe0dd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.089019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c83e3-a82c-46a2-8120-d14dcb40da19 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.091904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2d7c83e3-a82c-46a2-8120-d14dcb40da19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:48.121062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1253/2507] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:47 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:48.124770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651fe40-b7cc-45cb-8784-250c3edbe0dd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.139693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651fe40-b7cc-45cb-8784-250c3edbe0dd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.147620 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95bbf1-f993-4fce-9736-46250f53a832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.149081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95bbf1-f993-4fce-9736-46250f53a832 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.149493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95bbf1-f993-4fce-9736-46250f53a832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.149777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95bbf1-f993-4fce-9736-46250f53a832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.151057 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95bbf1-f993-4fce-9736-46250f53a832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.153065 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95bbf1-f993-4fce-9736-46250f53a832 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.167839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1254/2508] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:48.171631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95bbf1-f993-4fce-9736-46250f53a832 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.184980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95bbf1-f993-4fce-9736-46250f53a832 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.196747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e96cd0-f927-4ea9-b222-4e04ae6d0d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.198098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e96cd0-f927-4ea9-b222-4e04ae6d0d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.198769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e96cd0-f927-4ea9-b222-4e04ae6d0d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.199062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c83e3-a82c-46a2-8120-d14dcb40da19 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.199571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e96cd0-f927-4ea9-b222-4e04ae6d0d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.200780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e96cd0-f927-4ea9-b222-4e04ae6d0d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.202858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e96cd0-f927-4ea9-b222-4e04ae6d0d7d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.205899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbc18a-24d1-4be0-854a-735799870b7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee0fe35d9e304ee495998c93c6909fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.206605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbc18a-24d1-4be0-854a-735799870b7e service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.206882 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbc18a-24d1-4be0-854a-735799870b7e service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.207154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbc18a-24d1-4be0-854a-735799870b7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.213965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1255/2509] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:34:48.217782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbc18a-24d1-4be0-854a-735799870b7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.217954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e96cd0-f927-4ea9-b222-4e04ae6d0d7d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.219535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbc18a-24d1-4be0-854a-735799870b7e service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.228948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e96cd0-f927-4ea9-b222-4e04ae6d0d7d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.230385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1255/2510] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Sep 21 05:34:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 05:34:48.234235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9844e-5634-479f-b97e-ead86fa7fd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.234815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9844e-5634-479f-b97e-ead86fa7fd24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.235089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9844e-5634-479f-b97e-ead86fa7fd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.235363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9844e-5634-479f-b97e-ead86fa7fd24 admin admin] PATH_INFO: `/v3/projects/4277aa7fe0444fe2a06ab829c99ed5d4/users/08d79e81878a492f9a9833d988395d5f/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.253212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9844e-5634-479f-b97e-ead86fa7fd24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4277aa7fe0444fe2a06ab829c99ed5d4, user_id=08d79e81878a492f9a9833d988395d5f, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.256165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9844e-5634-479f-b97e-ead86fa7fd24 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.272823 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1256/2511] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:48 2022] PUT /identity/v3/projects/4277aa7fe0444fe2a06ab829c99ed5d4/users/08d79e81878a492f9a9833d988395d5f/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:48.283172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64500b03-bc18-405b-91fd-9614aef8e994 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.283469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64500b03-bc18-405b-91fd-9614aef8e994 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.283812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64500b03-bc18-405b-91fd-9614aef8e994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.310889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:48.321157 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-64500b03-bc18-405b-91fd-9614aef8e994 None None] MFA Rules not processed for user `08d79e81878a492f9a9833d988395d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:48.323247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-64500b03-bc18-405b-91fd-9614aef8e994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:48.414038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1256/2512] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:48.415459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9844e-5634-479f-b97e-ead86fa7fd24 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.436438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:48.493523 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9844e-5634-479f-b97e-ead86fa7fd24 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.500224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1514d-70c6-416b-9aa3-dbd10fbca8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.500953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1514d-70c6-416b-9aa3-dbd10fbca8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.501440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1514d-70c6-416b-9aa3-dbd10fbca8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.502215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1514d-70c6-416b-9aa3-dbd10fbca8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.502910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1514d-70c6-416b-9aa3-dbd10fbca8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.505624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1514d-70c6-416b-9aa3-dbd10fbca8e3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.540183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1257/2513] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:48 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:34:48.544414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64500b03-bc18-405b-91fd-9614aef8e994 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.564202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64500b03-bc18-405b-91fd-9614aef8e994 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.643010 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d0af0-6e17-454d-902e-5e76bb1ace14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.643745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d0af0-6e17-454d-902e-5e76bb1ace14 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.644131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d0af0-6e17-454d-902e-5e76bb1ace14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.644506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d0af0-6e17-454d-902e-5e76bb1ace14 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.650497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d0af0-6e17-454d-902e-5e76bb1ace14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.652674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d0af0-6e17-454d-902e-5e76bb1ace14 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.679538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1257/2514] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:48 2022] POST /identity/v3/users => generated 300 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:48.683999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1514d-70c6-416b-9aa3-dbd10fbca8e3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.696611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1514d-70c6-416b-9aa3-dbd10fbca8e3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.702717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e43ae4-a1b5-4564-a312-809180a7c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.703598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e43ae4-a1b5-4564-a312-809180a7c244 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.704045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e43ae4-a1b5-4564-a312-809180a7c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.704488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e43ae4-a1b5-4564-a312-809180a7c244 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.705284 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e43ae4-a1b5-4564-a312-809180a7c244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.706958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e43ae4-a1b5-4564-a312-809180a7c244 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.724018 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d0af0-6e17-454d-902e-5e76bb1ace14 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.724544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1258/2515] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:48.737240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d0af0-6e17-454d-902e-5e76bb1ace14 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.744037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec7fb-1b9f-475f-b78c-866c8922aa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.745245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec7fb-1b9f-475f-b78c-866c8922aa91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.745534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec7fb-1b9f-475f-b78c-866c8922aa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.745812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec7fb-1b9f-475f-b78c-866c8922aa91 admin admin] PATH_INFO: `/v3/projects/db608fbe28234789a56877d90c8cc2ba/users/39d175cbff7e4452bee680b2530dd185/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.764941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ec7fb-1b9f-475f-b78c-866c8922aa91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db608fbe28234789a56877d90c8cc2ba, user_id=39d175cbff7e4452bee680b2530dd185, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.768026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ec7fb-1b9f-475f-b78c-866c8922aa91 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.789920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1258/2516] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:48 2022] PUT /identity/v3/projects/db608fbe28234789a56877d90c8cc2ba/users/39d175cbff7e4452bee680b2530dd185/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:48.793640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e43ae4-a1b5-4564-a312-809180a7c244 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.806171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e43ae4-a1b5-4564-a312-809180a7c244 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.884068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112fef8-c65d-4a1f-ba16-fcc40437e7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.884784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112fef8-c65d-4a1f-ba16-fcc40437e7da admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.885136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112fef8-c65d-4a1f-ba16-fcc40437e7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.885477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112fef8-c65d-4a1f-ba16-fcc40437e7da admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.886516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112fef8-c65d-4a1f-ba16-fcc40437e7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.888573 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112fef8-c65d-4a1f-ba16-fcc40437e7da admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.903040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1259/2517] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:48.907291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec7fb-1b9f-475f-b78c-866c8922aa91 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:48.918845 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec7fb-1b9f-475f-b78c-866c8922aa91 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:48.924923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374f8b9-37a1-43f3-a307-347c7b52433b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:48.925669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374f8b9-37a1-43f3-a307-347c7b52433b admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.926053 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374f8b9-37a1-43f3-a307-347c7b52433b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.926336 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374f8b9-37a1-43f3-a307-347c7b52433b admin admin] PATH_INFO: `/v3/projects/db608fbe28234789a56877d90c8cc2ba/users/39d175cbff7e4452bee680b2530dd185/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:48.928495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374f8b9-37a1-43f3-a307-347c7b52433b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db608fbe28234789a56877d90c8cc2ba, user_id=39d175cbff7e4452bee680b2530dd185, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:48.932547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374f8b9-37a1-43f3-a307-347c7b52433b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:48.952182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1259/2518] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:48 2022] PUT /identity/v3/projects/db608fbe28234789a56877d90c8cc2ba/users/39d175cbff7e4452bee680b2530dd185/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:48.964333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52662abc-2549-4272-ba7d-a3b85ff74c58 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:48.966064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52662abc-2549-4272-ba7d-a3b85ff74c58 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:48.966064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52662abc-2549-4272-ba7d-a3b85ff74c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:49.008631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-52662abc-2549-4272-ba7d-a3b85ff74c58 None None] MFA Rules not processed for user `39d175cbff7e4452bee680b2530dd185`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:49.010509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-52662abc-2549-4272-ba7d-a3b85ff74c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:49.102388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1260/2519] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:49.115164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374f8b9-37a1-43f3-a307-347c7b52433b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:49.119121 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9374f8b9-37a1-43f3-a307-347c7b52433b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:49.214942 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374f8b9-37a1-43f3-a307-347c7b52433b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:49.221656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf9b59-87f1-4f2d-b5f9-537dd434afcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:49.222694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf9b59-87f1-4f2d-b5f9-537dd434afcc service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:49.222976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf9b59-87f1-4f2d-b5f9-537dd434afcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:49.223269 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf9b59-87f1-4f2d-b5f9-537dd434afcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:49.232368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:49.240384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf9b59-87f1-4f2d-b5f9-537dd434afcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:49.251233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf9b59-87f1-4f2d-b5f9-537dd434afcc service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:49.264349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1260/2520] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:49.887084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52662abc-2549-4272-ba7d-a3b85ff74c58 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:49.899839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:49.948977 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52662abc-2549-4272-ba7d-a3b85ff74c58 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:49.953377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:49.954022 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:49.954300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:49.954568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] PATH_INFO: `/v3/users/08d79e81878a492f9a9833d988395d5f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:49.955665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d79e81878a492f9a9833d988395d5f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:49.957576 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:50.004562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d79e81878a492f9a9833d988395d5f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.012857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d79e81878a492f9a9833d988395d5f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.013657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d79e81878a492f9a9833d988395d5f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.014283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d79e81878a492f9a9833d988395d5f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.021257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d79e81878a492f9a9833d988395d5f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.038776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1261/2521] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:49 2022] DELETE /identity/v3/users/08d79e81878a492f9a9833d988395d5f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:34:50.043762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37a18e-ce5d-4faa-a9a5-566da02c84e1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:50.044048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37a18e-ce5d-4faa-a9a5-566da02c84e1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:50.044320 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37a18e-ce5d-4faa-a9a5-566da02c84e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:50.062683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ea37a18e-ce5d-4faa-a9a5-566da02c84e1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:50.063927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ea37a18e-ce5d-4faa-a9a5-566da02c84e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:50.128645 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1261/2522] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:50.138245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:50.139735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:50.194809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa941-89c5-4094-8b59-b062af61e5d8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:50.198371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff8bb0-3537-4487-b584-334a363886cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:50.198962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff8bb0-3537-4487-b584-334a363886cc service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:50.199240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff8bb0-3537-4487-b584-334a363886cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:50.199510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff8bb0-3537-4487-b584-334a363886cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:50.208611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff8bb0-3537-4487-b584-334a363886cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:50.210048 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff8bb0-3537-4487-b584-334a363886cc service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:50.219854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1262/2523] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:34:50.492745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37a18e-ce5d-4faa-a9a5-566da02c84e1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:50.494315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ea37a18e-ce5d-4faa-a9a5-566da02c84e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:50.506404 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37a18e-ce5d-4faa-a9a5-566da02c84e1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:50.512699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:50.513483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:50.513839 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:50.514217 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] PATH_INFO: `/v3/projects/4277aa7fe0444fe2a06ab829c99ed5d4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:50.515405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4277aa7fe0444fe2a06ab829c99ed5d4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:50.517382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:50.568642 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4277aa7fe0444fe2a06ab829c99ed5d4', 'request_id': 'req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.575782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4277aa7fe0444fe2a06ab829c99ed5d4', 'request_id': 'req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.582990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4277aa7fe0444fe2a06ab829c99ed5d4', 'request_id': 'req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.602624 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] The token cache is being invalidate because project 4277aa7fe0444fe2a06ab829c99ed5d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:50.602838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.603489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.604980 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1262/2524] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:50 2022] DELETE /identity/v3/projects/4277aa7fe0444fe2a06ab829c99ed5d4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:50.608974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff8bb0-3537-4487-b584-334a363886cc service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:50.620148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff8bb0-3537-4487-b584-334a363886cc service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:50.649633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:50.676874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:50.677484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:50.677788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:50.678091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] PATH_INFO: `/v3/users/39d175cbff7e4452bee680b2530dd185` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:50.679195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d175cbff7e4452bee680b2530dd185)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:50.681640 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:50.726678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d175cbff7e4452bee680b2530dd185'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.733313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d175cbff7e4452bee680b2530dd185'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.733911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d175cbff7e4452bee680b2530dd185'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.734533 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d175cbff7e4452bee680b2530dd185'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.741612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b36de28f-5126-4eac-8d60-9a85393147f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d175cbff7e4452bee680b2530dd185'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:50.759222 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1263/2525] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:34:50 2022] DELETE /identity/v3/users/39d175cbff7e4452bee680b2530dd185 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:51.024722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:51.026221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:51.039250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd5bca-bc51-4043-8e7a-9361b9feb3f9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:51.095466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:51.096088 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.096363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.096632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] PATH_INFO: `/v3/projects/db608fbe28234789a56877d90c8cc2ba` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.097723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db608fbe28234789a56877d90c8cc2ba)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:51.099655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:51.121622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:51.144678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db608fbe28234789a56877d90c8cc2ba', 'request_id': 'req-e0c698dd-8630-49dd-8317-272dc756ce83'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:51.151588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db608fbe28234789a56877d90c8cc2ba', 'request_id': 'req-e0c698dd-8630-49dd-8317-272dc756ce83'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:51.157963 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db608fbe28234789a56877d90c8cc2ba', 'request_id': 'req-e0c698dd-8630-49dd-8317-272dc756ce83'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:51.168752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] The token cache is being invalidate because project db608fbe28234789a56877d90c8cc2ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:34:51.169041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:51.169683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0c698dd-8630-49dd-8317-272dc756ce83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:34:51.171189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1263/2526] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:34:51 2022] DELETE /identity/v3/projects/db608fbe28234789a56877d90c8cc2ba => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:51.176173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9136-fb8f-479e-a788-ae7fda34b341 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.176619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9136-fb8f-479e-a788-ae7fda34b341 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.176715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9136-fb8f-479e-a788-ae7fda34b341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.197588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d7fc9136-fb8f-479e-a788-ae7fda34b341 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:51.198805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d7fc9136-fb8f-479e-a788-ae7fda34b341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:51.256362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1264/2527] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:51.264511 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fff9a-6761-438d-9594-c9a75d8065d1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.264794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fff9a-6761-438d-9594-c9a75d8065d1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.265309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fff9a-6761-438d-9594-c9a75d8065d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.287355 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ea7fff9a-6761-438d-9594-c9a75d8065d1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:51.288563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ea7fff9a-6761-438d-9594-c9a75d8065d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:51.296575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1264/2528] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:34:51.300838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9136-fb8f-479e-a788-ae7fda34b341 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:51.318148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9136-fb8f-479e-a788-ae7fda34b341 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:51.318148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaadbd4-e815-4834-a3ab-ecf0032053da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:51.318148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaadbd4-e815-4834-a3ab-ecf0032053da admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.318148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaadbd4-e815-4834-a3ab-ecf0032053da admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.318538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaadbd4-e815-4834-a3ab-ecf0032053da admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.318538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaadbd4-e815-4834-a3ab-ecf0032053da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:34:51.318538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaadbd4-e815-4834-a3ab-ecf0032053da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:51.318816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaadbd4-e815-4834-a3ab-ecf0032053da admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:51.331143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1265/2529] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:34:51.335231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fff9a-6761-438d-9594-c9a75d8065d1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:51.345224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fff9a-6761-438d-9594-c9a75d8065d1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:51.350464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778937b-c93c-411a-82e4-64b1e071bff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:51.351074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778937b-c93c-411a-82e4-64b1e071bff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.351353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778937b-c93c-411a-82e4-64b1e071bff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.351626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778937b-c93c-411a-82e4-64b1e071bff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.352295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778937b-c93c-411a-82e4-64b1e071bff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:51.354221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778937b-c93c-411a-82e4-64b1e071bff4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:51.390754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1265/2530] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:34:51 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:51.394828 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaadbd4-e815-4834-a3ab-ecf0032053da admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:51.405375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaadbd4-e815-4834-a3ab-ecf0032053da admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:51.442965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:34:51.463311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f3a7e-b372-4c72-8914-01b07a8bc75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:51.463913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f3a7e-b372-4c72-8914-01b07a8bc75a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.464199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f3a7e-b372-4c72-8914-01b07a8bc75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.464473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f3a7e-b372-4c72-8914-01b07a8bc75a admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.465103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f3a7e-b372-4c72-8914-01b07a8bc75a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:51.467035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f3a7e-b372-4c72-8914-01b07a8bc75a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:51.486889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1266/2531] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:34:51 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:34:51.491131 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778937b-c93c-411a-82e4-64b1e071bff4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:51.501812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778937b-c93c-411a-82e4-64b1e071bff4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:51.506486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a73e75-09dd-49b6-9be0-8e737d77f054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:51.507107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a73e75-09dd-49b6-9be0-8e737d77f054 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.507411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a73e75-09dd-49b6-9be0-8e737d77f054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.507681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a73e75-09dd-49b6-9be0-8e737d77f054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.508917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a73e75-09dd-49b6-9be0-8e737d77f054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:51.510525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a73e75-09dd-49b6-9be0-8e737d77f054 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:51.520367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1266/2532] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:51.524157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f3a7e-b372-4c72-8914-01b07a8bc75a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:51.533858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f3a7e-b372-4c72-8914-01b07a8bc75a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:51.538474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923e86d-f859-47d6-aedf-1ef98da0ede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:51.539064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923e86d-f859-47d6-aedf-1ef98da0ede7 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.539394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923e86d-f859-47d6-aedf-1ef98da0ede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.539670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923e86d-f859-47d6-aedf-1ef98da0ede7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.540303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923e86d-f859-47d6-aedf-1ef98da0ede7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:51.541863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923e86d-f859-47d6-aedf-1ef98da0ede7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:51.551529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1267/2533] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:34:51.555858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a73e75-09dd-49b6-9be0-8e737d77f054 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:51.565628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a73e75-09dd-49b6-9be0-8e737d77f054 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:51.570232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b81f17-f5f3-4e22-809c-7444612fb6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:51.571437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81f17-f5f3-4e22-809c-7444612fb6ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.571714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81f17-f5f3-4e22-809c-7444612fb6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.571985 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81f17-f5f3-4e22-809c-7444612fb6ae admin admin] PATH_INFO: `/v3/projects/0fed1125901440369bbc9e030316d74b/users/20eeffa6b6a646df95117ecd0852a57b/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.587384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b81f17-f5f3-4e22-809c-7444612fb6ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fed1125901440369bbc9e030316d74b, user_id=20eeffa6b6a646df95117ecd0852a57b, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:51.590084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b81f17-f5f3-4e22-809c-7444612fb6ae admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:51.603125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1267/2534] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:34:51 2022] PUT /identity/v3/projects/0fed1125901440369bbc9e030316d74b/users/20eeffa6b6a646df95117ecd0852a57b/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:34:51.614692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb079-4a21-4eb4-b92e-afecef28c163 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.614978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb079-4a21-4eb4-b92e-afecef28c163 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.615253 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb079-4a21-4eb4-b92e-afecef28c163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.644227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9b9bb079-4a21-4eb4-b92e-afecef28c163 None None] MFA Rules not processed for user `20eeffa6b6a646df95117ecd0852a57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:34:51.645424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9b9bb079-4a21-4eb4-b92e-afecef28c163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:51.711400 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1268/2535] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:34:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:34:51.722856 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b81f17-f5f3-4e22-809c-7444612fb6ae admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:34:51.724723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-40b81f17-f5f3-4e22-809c-7444612fb6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:34:51.783886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b81f17-f5f3-4e22-809c-7444612fb6ae admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:34:51.787911 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e95ae6-a40b-4113-975e-43f1d7ac9291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:34:51.788512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e95ae6-a40b-4113-975e-43f1d7ac9291 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:34:51.788786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e95ae6-a40b-4113-975e-43f1d7ac9291 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:34:51.789103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e95ae6-a40b-4113-975e-43f1d7ac9291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:34:51.798282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e95ae6-a40b-4113-975e-43f1d7ac9291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:34:51.799687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e95ae6-a40b-4113-975e-43f1d7ac9291 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:34:51.808802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1268/2536] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:34:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:01.708300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:01.804397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=7,keystone:SELECT=68,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:02.362862 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb079-4a21-4eb4-b92e-afecef28c163 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:02.444844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb079-4a21-4eb4-b92e-afecef28c163 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:02.451024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:02.451763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:02.452046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:02.452316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] PATH_INFO: `/v3/users/20eeffa6b6a646df95117ecd0852a57b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:02.453485 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20eeffa6b6a646df95117ecd0852a57b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:02.455427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:02.512262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20eeffa6b6a646df95117ecd0852a57b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:02.521358 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20eeffa6b6a646df95117ecd0852a57b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:02.522079 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20eeffa6b6a646df95117ecd0852a57b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:02.523327 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20eeffa6b6a646df95117ecd0852a57b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:02.532687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20eeffa6b6a646df95117ecd0852a57b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:02.554729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1269/2537] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:02 2022] DELETE /identity/v3/users/20eeffa6b6a646df95117ecd0852a57b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:02.556974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268b19-0e00-4797-b3f8-b2b17b9cacf5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:02.557364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268b19-0e00-4797-b3f8-b2b17b9cacf5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:02.557740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268b19-0e00-4797-b3f8-b2b17b9cacf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:02.585410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3d268b19-0e00-4797-b3f8-b2b17b9cacf5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:02.587769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3d268b19-0e00-4797-b3f8-b2b17b9cacf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:02.661310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1269/2538] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:02.672663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:02.674204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:02.767769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c370bfe-7024-4779-b841-31b29a0c4293 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:02.773470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a610912-5a9f-4a8f-92fd-2b1ad64c79e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:02.774155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a610912-5a9f-4a8f-92fd-2b1ad64c79e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:02.774437 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a610912-5a9f-4a8f-92fd-2b1ad64c79e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:02.774748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a610912-5a9f-4a8f-92fd-2b1ad64c79e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:02.786418 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a610912-5a9f-4a8f-92fd-2b1ad64c79e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:02.787849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a610912-5a9f-4a8f-92fd-2b1ad64c79e5 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:02.798440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1270/2539] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:03.167201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268b19-0e00-4797-b3f8-b2b17b9cacf5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:03.169809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3d268b19-0e00-4797-b3f8-b2b17b9cacf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:03.189182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268b19-0e00-4797-b3f8-b2b17b9cacf5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:03.194477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:03.195126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.195447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.195883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] PATH_INFO: `/v3/projects/0fed1125901440369bbc9e030316d74b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.197324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fed1125901440369bbc9e030316d74b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:03.199451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:03.256432 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fed1125901440369bbc9e030316d74b', 'request_id': 'req-512091df-6d45-4875-aa1c-ebe9e32ad968'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:03.262609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fed1125901440369bbc9e030316d74b', 'request_id': 'req-512091df-6d45-4875-aa1c-ebe9e32ad968'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:03.271274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fed1125901440369bbc9e030316d74b', 'request_id': 'req-512091df-6d45-4875-aa1c-ebe9e32ad968'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:03.282874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] The token cache is being invalidate because project 0fed1125901440369bbc9e030316d74b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:03.283167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:03.283869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-512091df-6d45-4875-aa1c-ebe9e32ad968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:03.285367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1270/2540] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:03 2022] DELETE /identity/v3/projects/0fed1125901440369bbc9e030316d74b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:03.291019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31787598-040d-4984-a5d3-69cba10a8768 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.291408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31787598-040d-4984-a5d3-69cba10a8768 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.291780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31787598-040d-4984-a5d3-69cba10a8768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.314571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-31787598-040d-4984-a5d3-69cba10a8768 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:03.315709 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-31787598-040d-4984-a5d3-69cba10a8768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:03.333891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:03.386419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1271/2541] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:03.394423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de06a-8c94-4fa8-b563-2de5cdbcfeb4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.394822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de06a-8c94-4fa8-b563-2de5cdbcfeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.395168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de06a-8c94-4fa8-b563-2de5cdbcfeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.416916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9f0de06a-8c94-4fa8-b563-2de5cdbcfeb4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:03.418150 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9f0de06a-8c94-4fa8-b563-2de5cdbcfeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:03.425877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1271/2542] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:03.430066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31787598-040d-4984-a5d3-69cba10a8768 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:03.439916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31787598-040d-4984-a5d3-69cba10a8768 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:03.448125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4613b-e939-4c3b-aadf-7d9741b31b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:03.448490 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4613b-e939-4c3b-aadf-7d9741b31b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.448678 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4613b-e939-4c3b-aadf-7d9741b31b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.448955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4613b-e939-4c3b-aadf-7d9741b31b5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.449611 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4613b-e939-4c3b-aadf-7d9741b31b5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:03.449995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4613b-e939-4c3b-aadf-7d9741b31b5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:03.451524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4613b-e939-4c3b-aadf-7d9741b31b5f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:03.467925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1272/2543] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:03.469359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de06a-8c94-4fa8-b563-2de5cdbcfeb4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:03.479273 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de06a-8c94-4fa8-b563-2de5cdbcfeb4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:03.486563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c3088-0c61-40bd-846f-d8598bbdcac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:03.487182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c3088-0c61-40bd-846f-d8598bbdcac0 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.487441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c3088-0c61-40bd-846f-d8598bbdcac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.487720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c3088-0c61-40bd-846f-d8598bbdcac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.488362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c3088-0c61-40bd-846f-d8598bbdcac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:03.490486 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c3088-0c61-40bd-846f-d8598bbdcac0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:03.527079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1272/2544] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:03 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:03.528389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4613b-e939-4c3b-aadf-7d9741b31b5f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:03.540808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4613b-e939-4c3b-aadf-7d9741b31b5f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:03.618593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6305f-3f51-4a80-99f4-164d3b620347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:03.619117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6305f-3f51-4a80-99f4-164d3b620347 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.619401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6305f-3f51-4a80-99f4-164d3b620347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.619734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6305f-3f51-4a80-99f4-164d3b620347 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.620393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6305f-3f51-4a80-99f4-164d3b620347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:03.622495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6305f-3f51-4a80-99f4-164d3b620347 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:03.663229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1273/2545] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:03 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:03.664803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c3088-0c61-40bd-846f-d8598bbdcac0 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:03.677273 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c3088-0c61-40bd-846f-d8598bbdcac0 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:03.683787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d015c4b-38a4-4ba9-91fd-7062048e041b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:03.684500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d015c4b-38a4-4ba9-91fd-7062048e041b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.684858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d015c4b-38a4-4ba9-91fd-7062048e041b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.685134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d015c4b-38a4-4ba9-91fd-7062048e041b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.685785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d015c4b-38a4-4ba9-91fd-7062048e041b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:03.687460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d015c4b-38a4-4ba9-91fd-7062048e041b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:03.699296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1273/2546] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:03.703337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6305f-3f51-4a80-99f4-164d3b620347 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:03.714256 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6305f-3f51-4a80-99f4-164d3b620347 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:03.719067 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef6f15-b01b-48d0-9250-f39fe9313538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:03.719672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef6f15-b01b-48d0-9250-f39fe9313538 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.719955 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef6f15-b01b-48d0-9250-f39fe9313538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.720230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef6f15-b01b-48d0-9250-f39fe9313538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.720870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef6f15-b01b-48d0-9250-f39fe9313538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:03.722511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef6f15-b01b-48d0-9250-f39fe9313538 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:03.735503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1274/2547] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:03.739904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d015c4b-38a4-4ba9-91fd-7062048e041b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:03.750351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d015c4b-38a4-4ba9-91fd-7062048e041b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:03.755128 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064c43a-727f-4cd3-98ce-6246f24f829e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:03.755801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064c43a-727f-4cd3-98ce-6246f24f829e admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.756078 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064c43a-727f-4cd3-98ce-6246f24f829e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.756352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064c43a-727f-4cd3-98ce-6246f24f829e admin admin] PATH_INFO: `/v3/projects/8805a75f4d4e45e1ab7b6340160d5d44/users/c39060a2ae1441fb95086e8e6d1abe79/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.773829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064c43a-727f-4cd3-98ce-6246f24f829e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8805a75f4d4e45e1ab7b6340160d5d44, user_id=c39060a2ae1441fb95086e8e6d1abe79, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:03.777145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064c43a-727f-4cd3-98ce-6246f24f829e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:03.792833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1274/2548] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:03 2022] PUT /identity/v3/projects/8805a75f4d4e45e1ab7b6340160d5d44/users/c39060a2ae1441fb95086e8e6d1abe79/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:03.802854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46938-dca5-4675-aa07-56482f705823 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.803244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46938-dca5-4675-aa07-56482f705823 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:03.803449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46938-dca5-4675-aa07-56482f705823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:03.832519 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-acf46938-dca5-4675-aa07-56482f705823 None None] MFA Rules not processed for user `c39060a2ae1441fb95086e8e6d1abe79`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:03.833727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-acf46938-dca5-4675-aa07-56482f705823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:03.916353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1275/2549] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:03.920821 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064c43a-727f-4cd3-98ce-6246f24f829e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:03.947604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:03.993888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064c43a-727f-4cd3-98ce-6246f24f829e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:03.998952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4b4d1-9363-4319-ad92-584c5b793cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:03.999601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4b4d1-9363-4319-ad92-584c5b793cba admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:03.999891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4b4d1-9363-4319-ad92-584c5b793cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:04.000223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4b4d1-9363-4319-ad92-584c5b793cba admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:04.000944 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4b4d1-9363-4319-ad92-584c5b793cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:04.002934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4b4d1-9363-4319-ad92-584c5b793cba admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:04.030000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1275/2550] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:03 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:04.034231 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46938-dca5-4675-aa07-56482f705823 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:04.044756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46938-dca5-4675-aa07-56482f705823 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:04.057364 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:04.104580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c1872-dd52-4183-9686-f1c554ff30f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:04.105126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1872-dd52-4183-9686-f1c554ff30f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:04.105409 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1872-dd52-4183-9686-f1c554ff30f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:04.105686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1872-dd52-4183-9686-f1c554ff30f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:04.106362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1872-dd52-4183-9686-f1c554ff30f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:04.108250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1872-dd52-4183-9686-f1c554ff30f4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:04.129088 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1276/2551] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:04 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:04.133099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4b4d1-9363-4319-ad92-584c5b793cba admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:04.144392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4b4d1-9363-4319-ad92-584c5b793cba admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:04.149590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875febe-26d3-41c1-91b4-611e0836ce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:04.150420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875febe-26d3-41c1-91b4-611e0836ce66 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:04.150769 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875febe-26d3-41c1-91b4-611e0836ce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:04.151115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875febe-26d3-41c1-91b4-611e0836ce66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:04.151826 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875febe-26d3-41c1-91b4-611e0836ce66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:04.153435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875febe-26d3-41c1-91b4-611e0836ce66 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:04.166725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1276/2552] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:04.169847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c1872-dd52-4183-9686-f1c554ff30f4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:04.179624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c1872-dd52-4183-9686-f1c554ff30f4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:04.184812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b49595-2a73-4bd4-985b-4976b2912887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:04.185471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b49595-2a73-4bd4-985b-4976b2912887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:04.185752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b49595-2a73-4bd4-985b-4976b2912887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:04.186108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b49595-2a73-4bd4-985b-4976b2912887 admin admin] PATH_INFO: `/v3/projects/6aec42fa2fc947338ed687c1928792c8/users/ce52764c7290485da2d450dd1f691d17/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:04.203462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b49595-2a73-4bd4-985b-4976b2912887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aec42fa2fc947338ed687c1928792c8, user_id=ce52764c7290485da2d450dd1f691d17, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:04.206220 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b49595-2a73-4bd4-985b-4976b2912887 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:04.219852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1277/2553] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:04 2022] PUT /identity/v3/projects/6aec42fa2fc947338ed687c1928792c8/users/ce52764c7290485da2d450dd1f691d17/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:04.223757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875febe-26d3-41c1-91b4-611e0836ce66 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:04.234236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875febe-26d3-41c1-91b4-611e0836ce66 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:04.290737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab863d-6eed-4b92-aeda-6f26a41be6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:04.291370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab863d-6eed-4b92-aeda-6f26a41be6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:04.291651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab863d-6eed-4b92-aeda-6f26a41be6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:04.291922 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab863d-6eed-4b92-aeda-6f26a41be6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:04.292561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab863d-6eed-4b92-aeda-6f26a41be6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:04.294176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab863d-6eed-4b92-aeda-6f26a41be6a1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:04.304047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1277/2554] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:35:04.308953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b49595-2a73-4bd4-985b-4976b2912887 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:04.319448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b49595-2a73-4bd4-985b-4976b2912887 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:04.325211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4430e-7326-4ae7-bddf-515330826842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:04.326125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4430e-7326-4ae7-bddf-515330826842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:04.326408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4430e-7326-4ae7-bddf-515330826842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:04.326681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4430e-7326-4ae7-bddf-515330826842 admin admin] PATH_INFO: `/v3/projects/6aec42fa2fc947338ed687c1928792c8/users/ce52764c7290485da2d450dd1f691d17/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:04.328890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4430e-7326-4ae7-bddf-515330826842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aec42fa2fc947338ed687c1928792c8, user_id=ce52764c7290485da2d450dd1f691d17, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:04.332094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4430e-7326-4ae7-bddf-515330826842 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:04.346702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1278/2555] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:04 2022] PUT /identity/v3/projects/6aec42fa2fc947338ed687c1928792c8/users/ce52764c7290485da2d450dd1f691d17/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:04.418098 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907fb90-3d39-4265-a786-d9960e64d7d7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:04.418384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907fb90-3d39-4265-a786-d9960e64d7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:04.418665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907fb90-3d39-4265-a786-d9960e64d7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:04.447364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0907fb90-3d39-4265-a786-d9960e64d7d7 None None] MFA Rules not processed for user `ce52764c7290485da2d450dd1f691d17`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:04.448657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0907fb90-3d39-4265-a786-d9960e64d7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:04.519997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1278/2556] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:04.530816 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4430e-7326-4ae7-bddf-515330826842 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:04.532478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-09b4430e-7326-4ae7-bddf-515330826842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:04.592308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4430e-7326-4ae7-bddf-515330826842 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:04.596239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daecab9-550d-49ef-b400-c84e6eef6e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:04.596954 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daecab9-550d-49ef-b400-c84e6eef6e9f service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:04.597304 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daecab9-550d-49ef-b400-c84e6eef6e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:04.597649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daecab9-550d-49ef-b400-c84e6eef6e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:04.656227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daecab9-550d-49ef-b400-c84e6eef6e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:04.658482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daecab9-550d-49ef-b400-c84e6eef6e9f service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:04.674094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1279/2557] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:07.303829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907fb90-3d39-4265-a786-d9960e64d7d7 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:07.326349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:07.336154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daecab9-550d-49ef-b400-c84e6eef6e9f service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:07.338313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9daecab9-550d-49ef-b400-c84e6eef6e9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:07.374301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:07.389155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907fb90-3d39-4265-a786-d9960e64d7d7 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:07.394865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:07.395715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:07.396066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:07.396417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] PATH_INFO: `/v3/users/c39060a2ae1441fb95086e8e6d1abe79` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:07.397703 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39060a2ae1441fb95086e8e6d1abe79)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:07.399998 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:07.428305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daecab9-550d-49ef-b400-c84e6eef6e9f service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:07.434277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:07.435423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:07.436542 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:07.436968 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] PATH_INFO: `/v3/users/55a5e9bc6c5347b9988851a26aa09813` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:07.438146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a5e9bc6c5347b9988851a26aa09813)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:07.440045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:07.461749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39060a2ae1441fb95086e8e6d1abe79'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.462564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39060a2ae1441fb95086e8e6d1abe79'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.472816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39060a2ae1441fb95086e8e6d1abe79'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.474008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39060a2ae1441fb95086e8e6d1abe79'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.483739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-45e93599-778a-403a-90be-f908d87470d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39060a2ae1441fb95086e8e6d1abe79'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.495956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a5e9bc6c5347b9988851a26aa09813'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.503813 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a5e9bc6c5347b9988851a26aa09813'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.504551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a5e9bc6c5347b9988851a26aa09813'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.505286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a5e9bc6c5347b9988851a26aa09813'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.506215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1279/2558] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:07 2022] DELETE /identity/v3/users/c39060a2ae1441fb95086e8e6d1abe79 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:07.510541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e3753-1535-470e-a88c-737ad9d4d561 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:07.510820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e3753-1535-470e-a88c-737ad9d4d561 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:07.511429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e3753-1535-470e-a88c-737ad9d4d561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:07.513050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-445d56f6-c401-450c-8933-e5dcbc706915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a5e9bc6c5347b9988851a26aa09813'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:07.530335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-848e3753-1535-470e-a88c-737ad9d4d561 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:07.531574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-848e3753-1535-470e-a88c-737ad9d4d561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:07.534478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1280/2559] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:07 2022] DELETE /identity/v3/users/55a5e9bc6c5347b9988851a26aa09813 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:07.538651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a176105-5081-4a42-a599-dfe4f16ed785 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:07.538937 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a176105-5081-4a42-a599-dfe4f16ed785 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:07.539341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a176105-5081-4a42-a599-dfe4f16ed785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:07.559075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5a176105-5081-4a42-a599-dfe4f16ed785 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:07.560199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5a176105-5081-4a42-a599-dfe4f16ed785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:07.598564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1280/2560] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:07.608447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e3753-1535-470e-a88c-737ad9d4d561 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:07.610067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-848e3753-1535-470e-a88c-737ad9d4d561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:07.623657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1281/2561] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:07.633487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a176105-5081-4a42-a599-dfe4f16ed785 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:07.635015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5a176105-5081-4a42-a599-dfe4f16ed785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:07.651897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:07.674115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e3753-1535-470e-a88c-737ad9d4d561 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:07.678983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce04e6a-b54f-4a8d-b7f8-8c079172db22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:07.679631 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce04e6a-b54f-4a8d-b7f8-8c079172db22 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:07.679909 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce04e6a-b54f-4a8d-b7f8-8c079172db22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:07.680205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce04e6a-b54f-4a8d-b7f8-8c079172db22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:07.688730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:07.690209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce04e6a-b54f-4a8d-b7f8-8c079172db22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:07.691834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce04e6a-b54f-4a8d-b7f8-8c079172db22 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:07.702904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a176105-5081-4a42-a599-dfe4f16ed785 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:07.705616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1281/2562] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:35:07.708023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0581a-f501-41ca-8053-ab3cdb733822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:07.708647 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0581a-f501-41ca-8053-ab3cdb733822 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:07.708939 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0581a-f501-41ca-8053-ab3cdb733822 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:07.710432 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0581a-f501-41ca-8053-ab3cdb733822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:07.721208 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0581a-f501-41ca-8053-ab3cdb733822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:07.722691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0581a-f501-41ca-8053-ab3cdb733822 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:07.734020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1282/2563] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:35:08.051095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce04e6a-b54f-4a8d-b7f8-8c079172db22 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.052625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8ce04e6a-b54f-4a8d-b7f8-8c079172db22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.072061 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0581a-f501-41ca-8053-ab3cdb733822 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.073959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-62f0581a-f501-41ca-8053-ab3cdb733822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.080943 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce04e6a-b54f-4a8d-b7f8-8c079172db22 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.089142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0581a-f501-41ca-8053-ab3cdb733822 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.090306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.090954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.091266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.091507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] PATH_INFO: `/v3/projects/8805a75f4d4e45e1ab7b6340160d5d44` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.092642 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8805a75f4d4e45e1ab7b6340160d5d44)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.094619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.097393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.098344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.098624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.098894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] PATH_INFO: `/v3/projects/b8710b7a992a463db3f8d1e78e784b77` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.100205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8710b7a992a463db3f8d1e78e784b77)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.102215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.154164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8805a75f4d4e45e1ab7b6340160d5d44', 'request_id': 'req-bb9373a9-9b6b-47a5-bc36-b7ede8526064'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.161650 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8805a75f4d4e45e1ab7b6340160d5d44', 'request_id': 'req-bb9373a9-9b6b-47a5-bc36-b7ede8526064'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.161860 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8710b7a992a463db3f8d1e78e784b77', 'request_id': 'req-dd918f22-c56f-4a77-8043-887814a25ed3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.170802 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8710b7a992a463db3f8d1e78e784b77', 'request_id': 'req-dd918f22-c56f-4a77-8043-887814a25ed3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.173757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8805a75f4d4e45e1ab7b6340160d5d44', 'request_id': 'req-bb9373a9-9b6b-47a5-bc36-b7ede8526064'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.178277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8710b7a992a463db3f8d1e78e784b77', 'request_id': 'req-dd918f22-c56f-4a77-8043-887814a25ed3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.190229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] The token cache is being invalidate because project b8710b7a992a463db3f8d1e78e784b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:08.190521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.191190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dd918f22-c56f-4a77-8043-887814a25ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.191636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] The token cache is being invalidate because project 8805a75f4d4e45e1ab7b6340160d5d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:08.192095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.192689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1283/2564] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:08 2022] DELETE /identity/v3/projects/b8710b7a992a463db3f8d1e78e784b77 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:08.196627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.197658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130c4c1-c421-43c4-9c41-aa4d411ed808 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.198063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130c4c1-c421-43c4-9c41-aa4d411ed808 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.198394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130c4c1-c421-43c4-9c41-aa4d411ed808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.201433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1282/2565] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:08 2022] DELETE /identity/v3/projects/8805a75f4d4e45e1ab7b6340160d5d44 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:08.202438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.203936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.221730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7130c4c1-c421-43c4-9c41-aa4d411ed808 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:08.223306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7130c4c1-c421-43c4-9c41-aa4d411ed808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.280111 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9373a9-9b6b-47a5-bc36-b7ede8526064 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.289894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1284/2566] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:08.345664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.346352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.346628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.346902 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] PATH_INFO: `/v3/users/ce52764c7290485da2d450dd1f691d17` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.348271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce52764c7290485da2d450dd1f691d17)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.350535 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.357009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f32ea-b417-4ddb-9fc3-b18e2c0ed42e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.357534 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f32ea-b417-4ddb-9fc3-b18e2c0ed42e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.357637 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f32ea-b417-4ddb-9fc3-b18e2c0ed42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.367897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:08.379804 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-693f32ea-b417-4ddb-9fc3-b18e2c0ed42e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:08.381015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-693f32ea-b417-4ddb-9fc3-b18e2c0ed42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.392395 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1285/2567] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:08.397074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f32ea-b417-4ddb-9fc3-b18e2c0ed42e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.409159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f32ea-b417-4ddb-9fc3-b18e2c0ed42e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.414939 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.415657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.416021 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.416357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.417063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:08.417441 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.419123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.422755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce52764c7290485da2d450dd1f691d17'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.423422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce52764c7290485da2d450dd1f691d17'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.430236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce52764c7290485da2d450dd1f691d17'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.430996 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce52764c7290485da2d450dd1f691d17'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.431427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1286/2568] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:35:08.435891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.437551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.438391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce52764c7290485da2d450dd1f691d17'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:08.450681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4aace-1b80-4416-8538-29351fe936fd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.455281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b3020-02aa-4be3-9f90-315c6502ff4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.455866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b3020-02aa-4be3-9f90-315c6502ff4b admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.456150 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b3020-02aa-4be3-9f90-315c6502ff4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.456423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b3020-02aa-4be3-9f90-315c6502ff4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.456588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1283/2569] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:08 2022] DELETE /identity/v3/users/ce52764c7290485da2d450dd1f691d17 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:08.459479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b3020-02aa-4be3-9f90-315c6502ff4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.459479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b3020-02aa-4be3-9f90-315c6502ff4b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.487585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1287/2570] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:08 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:35:08.491862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.571934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ed058-0b42-40bf-9a23-b44d4f869191 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.625592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b3020-02aa-4be3-9f90-315c6502ff4b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.627937 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-619b3020-02aa-4be3-9f90-315c6502ff4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.653764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290558c-7125-4d58-86f1-811043f81d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.654482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290558c-7125-4d58-86f1-811043f81d1d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.654761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290558c-7125-4d58-86f1-811043f81d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.655039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290558c-7125-4d58-86f1-811043f81d1d admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.655685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290558c-7125-4d58-86f1-811043f81d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.657602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290558c-7125-4d58-86f1-811043f81d1d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.664967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:08.687176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1284/2571] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:08 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:35:08.688138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290558c-7125-4d58-86f1-811043f81d1d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.698840 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b3020-02aa-4be3-9f90-315c6502ff4b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.699626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290558c-7125-4d58-86f1-811043f81d1d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.702346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8aa33-1ad5-4155-986d-fcbc4ca0da56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.702927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8aa33-1ad5-4155-986d-fcbc4ca0da56 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.703214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8aa33-1ad5-4155-986d-fcbc4ca0da56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.703477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8aa33-1ad5-4155-986d-fcbc4ca0da56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.705528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2ef4f-dd85-4875-bca1-d4a50ca9d9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.705668 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-62f8aa33-1ad5-4155-986d-fcbc4ca0da56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.705964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ef4f-dd85-4875-bca1-d4a50ca9d9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.706349 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ef4f-dd85-4875-bca1-d4a50ca9d9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.706652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ef4f-dd85-4875-bca1-d4a50ca9d9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.707430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2ef4f-dd85-4875-bca1-d4a50ca9d9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.709085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2ef4f-dd85-4875-bca1-d4a50ca9d9a0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.718118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8aa33-1ad5-4155-986d-fcbc4ca0da56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.719530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8aa33-1ad5-4155-986d-fcbc4ca0da56 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.720876 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1285/2572] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:08.725435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2ef4f-dd85-4875-bca1-d4a50ca9d9a0 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.729978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1288/2573] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:35:08.736440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2ef4f-dd85-4875-bca1-d4a50ca9d9a0 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.741333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c9487-9ac1-4cd3-a6ca-d91ff7cd5b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.742067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c9487-9ac1-4cd3-a6ca-d91ff7cd5b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.744514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c9487-9ac1-4cd3-a6ca-d91ff7cd5b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.744514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c9487-9ac1-4cd3-a6ca-d91ff7cd5b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.744514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c9487-9ac1-4cd3-a6ca-d91ff7cd5b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.745640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c9487-9ac1-4cd3-a6ca-d91ff7cd5b97 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.761741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1286/2574] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:08.762852 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8aa33-1ad5-4155-986d-fcbc4ca0da56 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.772733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8aa33-1ad5-4155-986d-fcbc4ca0da56 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.777657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045f85d-f355-48ad-99bf-b92ad5441ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.778394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045f85d-f355-48ad-99bf-b92ad5441ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.778616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045f85d-f355-48ad-99bf-b92ad5441ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.778890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045f85d-f355-48ad-99bf-b92ad5441ef7 admin admin] PATH_INFO: `/v3/projects/9e967d18b1214844af2c4a7a1bd22c67/users/53cc6ddee574442486f78d9ed033cd2e/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.795259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045f85d-f355-48ad-99bf-b92ad5441ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e967d18b1214844af2c4a7a1bd22c67, user_id=53cc6ddee574442486f78d9ed033cd2e, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.798201 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045f85d-f355-48ad-99bf-b92ad5441ef7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.811242 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1289/2575] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:08 2022] PUT /identity/v3/projects/9e967d18b1214844af2c4a7a1bd22c67/users/53cc6ddee574442486f78d9ed033cd2e/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:08.822047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461145-ba49-4716-9866-d297bda4847b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.822394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461145-ba49-4716-9866-d297bda4847b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.822564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461145-ba49-4716-9866-d297bda4847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.830652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:08.833171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045f85d-f355-48ad-99bf-b92ad5441ef7 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.834991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5045f85d-f355-48ad-99bf-b92ad5441ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.861512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-07461145-ba49-4716-9866-d297bda4847b None None] MFA Rules not processed for user `53cc6ddee574442486f78d9ed033cd2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:08.863391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-07461145-ba49-4716-9866-d297bda4847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.924746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045f85d-f355-48ad-99bf-b92ad5441ef7 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:08.929599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:08.930269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:08.930551 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:08.931387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] PATH_INFO: `/v3/projects/6aec42fa2fc947338ed687c1928792c8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:08.932525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aec42fa2fc947338ed687c1928792c8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:08.934471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:08.945151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1287/2576] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:08 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:08.957361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07461145-ba49-4716-9866-d297bda4847b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:08.958812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-07461145-ba49-4716-9866-d297bda4847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:08.997033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aec42fa2fc947338ed687c1928792c8', 'request_id': 'req-5434c664-9fb4-4c89-8322-08a1f8ecab59'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:09.006015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aec42fa2fc947338ed687c1928792c8', 'request_id': 'req-5434c664-9fb4-4c89-8322-08a1f8ecab59'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:09.012261 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aec42fa2fc947338ed687c1928792c8', 'request_id': 'req-5434c664-9fb4-4c89-8322-08a1f8ecab59'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:09.024845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] The token cache is being invalidate because project 6aec42fa2fc947338ed687c1928792c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:09.025275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:09.025979 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5434c664-9fb4-4c89-8322-08a1f8ecab59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:09.027021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07461145-ba49-4716-9866-d297bda4847b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:09.027810 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1290/2577] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:08 2022] DELETE /identity/v3/projects/6aec42fa2fc947338ed687c1928792c8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:09.032193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa59db9-f550-4a1b-8ef0-dc42cc87c891 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.032488 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa59db9-f550-4a1b-8ef0-dc42cc87c891 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.032763 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa59db9-f550-4a1b-8ef0-dc42cc87c891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.057317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1aa59db9-f550-4a1b-8ef0-dc42cc87c891 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:09.058606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1aa59db9-f550-4a1b-8ef0-dc42cc87c891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:09.081712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d1864-008d-48b6-b00e-d55b6f192fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:09.082334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d1864-008d-48b6-b00e-d55b6f192fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.082609 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d1864-008d-48b6-b00e-d55b6f192fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.082886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d1864-008d-48b6-b00e-d55b6f192fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.084682 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-209d1864-008d-48b6-b00e-d55b6f192fe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:09.092157 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:09.133241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1291/2578] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:09.140894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57d9ee-ff6c-469c-8514-52f7b270d4fb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.141190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57d9ee-ff6c-469c-8514-52f7b270d4fb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.141613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57d9ee-ff6c-469c-8514-52f7b270d4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.157437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d1864-008d-48b6-b00e-d55b6f192fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:09.158918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d1864-008d-48b6-b00e-d55b6f192fe5 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:09.163619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ae57d9ee-ff6c-469c-8514-52f7b270d4fb None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:09.164846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ae57d9ee-ff6c-469c-8514-52f7b270d4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:09.169434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1288/2579] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:08 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:35:09.174005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1292/2580] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:09.178343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d1864-008d-48b6-b00e-d55b6f192fe5 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:09.184526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:09.192740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d1864-008d-48b6-b00e-d55b6f192fe5 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:09.199266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf3e31-f25a-4906-8deb-ffed4ccb9170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:09.199987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf3e31-f25a-4906-8deb-ffed4ccb9170 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.200362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf3e31-f25a-4906-8deb-ffed4ccb9170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.200755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf3e31-f25a-4906-8deb-ffed4ccb9170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.201513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf3e31-f25a-4906-8deb-ffed4ccb9170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:09.202029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf3e31-f25a-4906-8deb-ffed4ccb9170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:09.203834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf3e31-f25a-4906-8deb-ffed4ccb9170 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:09.217994 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1289/2581] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:09.221712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57d9ee-ff6c-469c-8514-52f7b270d4fb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:09.232082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57d9ee-ff6c-469c-8514-52f7b270d4fb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:09.237250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c287036-e59c-4885-9fea-1112e264500e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:09.237876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c287036-e59c-4885-9fea-1112e264500e admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.238213 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c287036-e59c-4885-9fea-1112e264500e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.238486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c287036-e59c-4885-9fea-1112e264500e admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.239132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c287036-e59c-4885-9fea-1112e264500e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:09.241026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c287036-e59c-4885-9fea-1112e264500e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:09.276050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1293/2582] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:09 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:09.277395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf3e31-f25a-4906-8deb-ffed4ccb9170 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:09.293508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf3e31-f25a-4906-8deb-ffed4ccb9170 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:09.369384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7688e17-cc49-40ed-a112-714c2124e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:09.370076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7688e17-cc49-40ed-a112-714c2124e1ba admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.370354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7688e17-cc49-40ed-a112-714c2124e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.370629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7688e17-cc49-40ed-a112-714c2124e1ba admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.371300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7688e17-cc49-40ed-a112-714c2124e1ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:09.373253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7688e17-cc49-40ed-a112-714c2124e1ba admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:09.400329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1290/2583] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:09 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:09.401610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c287036-e59c-4885-9fea-1112e264500e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:09.412788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c287036-e59c-4885-9fea-1112e264500e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:09.420141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38385b4b-b86d-4ee4-9e10-23a5b5165d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:09.420987 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385b4b-b86d-4ee4-9e10-23a5b5165d3c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.421330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385b4b-b86d-4ee4-9e10-23a5b5165d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.421672 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385b4b-b86d-4ee4-9e10-23a5b5165d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.422617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38385b4b-b86d-4ee4-9e10-23a5b5165d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:09.424277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38385b4b-b86d-4ee4-9e10-23a5b5165d3c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:09.611348 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1294/2584] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:09.615692 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7688e17-cc49-40ed-a112-714c2124e1ba admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:09.629989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7688e17-cc49-40ed-a112-714c2124e1ba admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:09.635039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406d5b2-9491-4e29-ba55-8f3eba2165ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:09.635768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406d5b2-9491-4e29-ba55-8f3eba2165ed admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.636113 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406d5b2-9491-4e29-ba55-8f3eba2165ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.636449 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406d5b2-9491-4e29-ba55-8f3eba2165ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.637683 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406d5b2-9491-4e29-ba55-8f3eba2165ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:09.639526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406d5b2-9491-4e29-ba55-8f3eba2165ed admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:09.672627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1291/2585] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:09.677018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38385b4b-b86d-4ee4-9e10-23a5b5165d3c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:09.688144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38385b4b-b86d-4ee4-9e10-23a5b5165d3c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:09.693706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6228b-e80c-4804-aa4b-08236517e277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:09.694476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6228b-e80c-4804-aa4b-08236517e277 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.695643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6228b-e80c-4804-aa4b-08236517e277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.696117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6228b-e80c-4804-aa4b-08236517e277 admin admin] PATH_INFO: `/v3/projects/45fa304217f34292b0e0c8c02d9c0c39/users/f0e82f63e0ef4498b8d877d0ad9378c7/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.714211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6228b-e80c-4804-aa4b-08236517e277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fa304217f34292b0e0c8c02d9c0c39, user_id=f0e82f63e0ef4498b8d877d0ad9378c7, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:09.718903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6228b-e80c-4804-aa4b-08236517e277 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:09.737134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1295/2586] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:09 2022] PUT /identity/v3/projects/45fa304217f34292b0e0c8c02d9c0c39/users/f0e82f63e0ef4498b8d877d0ad9378c7/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:09.747975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159d517-0059-4771-a36d-2da0be7555c1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.748264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159d517-0059-4771-a36d-2da0be7555c1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.748537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159d517-0059-4771-a36d-2da0be7555c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.788396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d159d517-0059-4771-a36d-2da0be7555c1 None None] MFA Rules not processed for user `f0e82f63e0ef4498b8d877d0ad9378c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:09.789744 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d159d517-0059-4771-a36d-2da0be7555c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:09.856323 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1292/2587] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:09.868152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6228b-e80c-4804-aa4b-08236517e277 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:09.870065 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f5a6228b-e80c-4804-aa4b-08236517e277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:09.927289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6228b-e80c-4804-aa4b-08236517e277 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:09.931138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb59e7f-4a2a-4824-a6ae-9dd5d6e4c25e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:09.931903 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb59e7f-4a2a-4824-a6ae-9dd5d6e4c25e service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:09.932188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb59e7f-4a2a-4824-a6ae-9dd5d6e4c25e service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:09.932457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb59e7f-4a2a-4824-a6ae-9dd5d6e4c25e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:09.940873 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb59e7f-4a2a-4824-a6ae-9dd5d6e4c25e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:09.942324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb59e7f-4a2a-4824-a6ae-9dd5d6e4c25e service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:09.951951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1296/2588] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:09.978409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159d517-0059-4771-a36d-2da0be7555c1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:10.040571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:10.045010 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159d517-0059-4771-a36d-2da0be7555c1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:10.050507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:10.051138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:10.051415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:10.051686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] PATH_INFO: `/v3/users/f0e82f63e0ef4498b8d877d0ad9378c7` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:10.052819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0e82f63e0ef4498b8d877d0ad9378c7)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:10.054779 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:10.107120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0e82f63e0ef4498b8d877d0ad9378c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.107880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0e82f63e0ef4498b8d877d0ad9378c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.117762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0e82f63e0ef4498b8d877d0ad9378c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.119145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0e82f63e0ef4498b8d877d0ad9378c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.127916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0e82f63e0ef4498b8d877d0ad9378c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.152718 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1293/2589] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:09 2022] DELETE /identity/v3/users/f0e82f63e0ef4498b8d877d0ad9378c7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:10.157091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc77c6-da05-40ce-8763-7269c5bd7f2c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:10.157381 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc77c6-da05-40ce-8763-7269c5bd7f2c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:10.157661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc77c6-da05-40ce-8763-7269c5bd7f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:10.183275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2cfc77c6-da05-40ce-8763-7269c5bd7f2c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:10.184770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2cfc77c6-da05-40ce-8763-7269c5bd7f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:10.253143 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1297/2590] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:10.263834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:10.265518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:10.329806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8ffa-34db-4065-bc55-c4dda692f16b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:10.334643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423a75e-404c-46ef-92a3-0574b8d70354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:10.335435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423a75e-404c-46ef-92a3-0574b8d70354 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:10.335713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423a75e-404c-46ef-92a3-0574b8d70354 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:10.336061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423a75e-404c-46ef-92a3-0574b8d70354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:10.346506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423a75e-404c-46ef-92a3-0574b8d70354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:10.348153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423a75e-404c-46ef-92a3-0574b8d70354 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:10.359773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1294/2591] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:10.775199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc77c6-da05-40ce-8763-7269c5bd7f2c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:10.776631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2cfc77c6-da05-40ce-8763-7269c5bd7f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:10.792356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc77c6-da05-40ce-8763-7269c5bd7f2c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:10.805310 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:10.806560 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:10.806845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:10.808419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] PATH_INFO: `/v3/projects/45fa304217f34292b0e0c8c02d9c0c39` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:10.810902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fa304217f34292b0e0c8c02d9c0c39)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:10.812879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:10.840301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:10.894413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fa304217f34292b0e0c8c02d9c0c39', 'request_id': 'req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.908072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fa304217f34292b0e0c8c02d9c0c39', 'request_id': 'req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.921625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fa304217f34292b0e0c8c02d9c0c39', 'request_id': 'req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.938974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] The token cache is being invalidate because project 45fa304217f34292b0e0c8c02d9c0c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:10.939563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.941952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5a26b5e7-684b-4bdb-9bdc-e00371ca35a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:10.944098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1298/2592] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:10 2022] DELETE /identity/v3/projects/45fa304217f34292b0e0c8c02d9c0c39 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:10.949114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc8ccd-7df2-4fbb-aebb-9ac2e26c971d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:10.950009 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc8ccd-7df2-4fbb-aebb-9ac2e26c971d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:10.950461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc8ccd-7df2-4fbb-aebb-9ac2e26c971d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:10.975867 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8bdc8ccd-7df2-4fbb-aebb-9ac2e26c971d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:10.978264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8bdc8ccd-7df2-4fbb-aebb-9ac2e26c971d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:11.074146 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1295/2593] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:11.079092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1fef0-61a9-477c-9070-eb19fd93a1eb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:11.079716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1fef0-61a9-477c-9070-eb19fd93a1eb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:11.079716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1fef0-61a9-477c-9070-eb19fd93a1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:11.109086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b7c1fef0-61a9-477c-9070-eb19fd93a1eb None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:11.110843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b7c1fef0-61a9-477c-9070-eb19fd93a1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:11.120607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1299/2594] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:11.124797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc8ccd-7df2-4fbb-aebb-9ac2e26c971d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:11.145275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc8ccd-7df2-4fbb-aebb-9ac2e26c971d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:11.151801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea1270-ff04-445f-b2d1-def03e34505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:11.152959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea1270-ff04-445f-b2d1-def03e34505e admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:11.153350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea1270-ff04-445f-b2d1-def03e34505e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:11.153719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea1270-ff04-445f-b2d1-def03e34505e admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:11.154765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea1270-ff04-445f-b2d1-def03e34505e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:11.155309 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea1270-ff04-445f-b2d1-def03e34505e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:11.157340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea1270-ff04-445f-b2d1-def03e34505e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:11.172408 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1296/2595] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:11.176895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1fef0-61a9-477c-9070-eb19fd93a1eb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:11.187911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1fef0-61a9-477c-9070-eb19fd93a1eb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:11.193245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d0d85-f242-4071-aa83-6a58d457f53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:11.194026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d0d85-f242-4071-aa83-6a58d457f53e admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:11.194412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d0d85-f242-4071-aa83-6a58d457f53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:11.194788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d0d85-f242-4071-aa83-6a58d457f53e admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:11.195531 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d0d85-f242-4071-aa83-6a58d457f53e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:11.197986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d0d85-f242-4071-aa83-6a58d457f53e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:11.237682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1300/2596] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:11 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:11.241702 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea1270-ff04-445f-b2d1-def03e34505e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:11.252088 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea1270-ff04-445f-b2d1-def03e34505e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:11.256021 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:11.315410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fab53b-1e61-46c7-9270-c4ed74b78cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:11.316331 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab53b-1e61-46c7-9270-c4ed74b78cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:11.316796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab53b-1e61-46c7-9270-c4ed74b78cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:11.317077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab53b-1e61-46c7-9270-c4ed74b78cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:11.317970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fab53b-1e61-46c7-9270-c4ed74b78cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:11.320218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fab53b-1e61-46c7-9270-c4ed74b78cc4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:11.344652 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1297/2597] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:11 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:11.349818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d0d85-f242-4071-aa83-6a58d457f53e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:11.363846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d0d85-f242-4071-aa83-6a58d457f53e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:11.370406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13087f-a0e7-43e6-adb0-1c6de4ae13c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:11.371137 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13087f-a0e7-43e6-adb0-1c6de4ae13c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:11.371524 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13087f-a0e7-43e6-adb0-1c6de4ae13c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:11.371901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13087f-a0e7-43e6-adb0-1c6de4ae13c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:11.372658 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13087f-a0e7-43e6-adb0-1c6de4ae13c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:11.374438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13087f-a0e7-43e6-adb0-1c6de4ae13c5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:11.387885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1301/2598] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:35:11.389166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fab53b-1e61-46c7-9270-c4ed74b78cc4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:11.399680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fab53b-1e61-46c7-9270-c4ed74b78cc4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:11.405416 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6f082-7461-4a18-8033-e12c7b722d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:11.406071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6f082-7461-4a18-8033-e12c7b722d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:11.406515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6f082-7461-4a18-8033-e12c7b722d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:11.406690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6f082-7461-4a18-8033-e12c7b722d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:11.407520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6f082-7461-4a18-8033-e12c7b722d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:11.409348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6f082-7461-4a18-8033-e12c7b722d6a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:11.436415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1298/2599] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:11.441314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13087f-a0e7-43e6-adb0-1c6de4ae13c5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:11.452331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13087f-a0e7-43e6-adb0-1c6de4ae13c5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:11.457495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab29ef-f8f9-4c8f-b5b7-eb7c1f4be2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:11.458269 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab29ef-f8f9-4c8f-b5b7-eb7c1f4be2ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:11.458768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab29ef-f8f9-4c8f-b5b7-eb7c1f4be2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:11.459113 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab29ef-f8f9-4c8f-b5b7-eb7c1f4be2ab admin admin] PATH_INFO: `/v3/projects/20e18203e3fe44feab8f9afbba026f3d/users/d388f9040af147229dbeb3e46353b49d/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:11.477924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab29ef-f8f9-4c8f-b5b7-eb7c1f4be2ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e18203e3fe44feab8f9afbba026f3d, user_id=d388f9040af147229dbeb3e46353b49d, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:11.480839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab29ef-f8f9-4c8f-b5b7-eb7c1f4be2ab admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:11.496536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1302/2600] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:11 2022] PUT /identity/v3/projects/20e18203e3fe44feab8f9afbba026f3d/users/d388f9040af147229dbeb3e46353b49d/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:11.507530 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f2c42-1cb7-4bb8-b6f7-9bb68f0d97a1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:11.507885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f2c42-1cb7-4bb8-b6f7-9bb68f0d97a1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:11.508334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f2c42-1cb7-4bb8-b6f7-9bb68f0d97a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:11.538272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4f7f2c42-1cb7-4bb8-b6f7-9bb68f0d97a1 None None] MFA Rules not processed for user `d388f9040af147229dbeb3e46353b49d`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:11.539710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4f7f2c42-1cb7-4bb8-b6f7-9bb68f0d97a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:11.609265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1299/2601] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:11.622496 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab29ef-f8f9-4c8f-b5b7-eb7c1f4be2ab admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:11.624350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-daab29ef-f8f9-4c8f-b5b7-eb7c1f4be2ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:11.683942 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab29ef-f8f9-4c8f-b5b7-eb7c1f4be2ab admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:11.687581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a7101-807e-4741-bb28-6ecd68ddbc5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:11.688180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a7101-807e-4741-bb28-6ecd68ddbc5e service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:11.688461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a7101-807e-4741-bb28-6ecd68ddbc5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:11.688751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a7101-807e-4741-bb28-6ecd68ddbc5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:11.698248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a7101-807e-4741-bb28-6ecd68ddbc5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:11.699957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a7101-807e-4741-bb28-6ecd68ddbc5e service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:11.711449 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1303/2602] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:12.963025 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f2c42-1cb7-4bb8-b6f7-9bb68f0d97a1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:12.965082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4f7f2c42-1cb7-4bb8-b6f7-9bb68f0d97a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:13.049789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f2c42-1cb7-4bb8-b6f7-9bb68f0d97a1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:13.054499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:13.055164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.055444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.055749 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] PATH_INFO: `/v3/users/1ec24729b04947b7b05cc6dcd787eb9a` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:13.057007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ec24729b04947b7b05cc6dcd787eb9a)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:13.059099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:13.076240 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:13.109433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec24729b04947b7b05cc6dcd787eb9a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.110343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec24729b04947b7b05cc6dcd787eb9a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.123644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec24729b04947b7b05cc6dcd787eb9a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.123644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec24729b04947b7b05cc6dcd787eb9a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.129152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33109707-1ae4-46cc-94b2-06d71111dd59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec24729b04947b7b05cc6dcd787eb9a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.147390 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1300/2603] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:12 2022] DELETE /identity/v3/users/1ec24729b04947b7b05cc6dcd787eb9a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:13.356886 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a7101-807e-4741-bb28-6ecd68ddbc5e service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:13.373624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a7101-807e-4741-bb28-6ecd68ddbc5e service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:13.416839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:13.438857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:13.439487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.439774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.440042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] PATH_INFO: `/v3/projects/53b1fb5bb9cd41959ebca2804fb426a2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:13.441155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b1fb5bb9cd41959ebca2804fb426a2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:13.443121 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:13.493696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b1fb5bb9cd41959ebca2804fb426a2', 'request_id': 'req-1fc6a784-4687-4332-afbd-749e10df7436'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.501594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b1fb5bb9cd41959ebca2804fb426a2', 'request_id': 'req-1fc6a784-4687-4332-afbd-749e10df7436'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.508574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b1fb5bb9cd41959ebca2804fb426a2', 'request_id': 'req-1fc6a784-4687-4332-afbd-749e10df7436'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.519785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] The token cache is being invalidate because project 53b1fb5bb9cd41959ebca2804fb426a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:13.520089 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.520724 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1fc6a784-4687-4332-afbd-749e10df7436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:13.522204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1304/2604] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:13 2022] DELETE /identity/v3/projects/53b1fb5bb9cd41959ebca2804fb426a2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:13.528347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d73e5-5fc8-4d84-a5ac-3357ec94fa52 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.528496 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d73e5-5fc8-4d84-a5ac-3357ec94fa52 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.528899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d73e5-5fc8-4d84-a5ac-3357ec94fa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:13.552992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ee3d73e5-5fc8-4d84-a5ac-3357ec94fa52 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:13.554376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ee3d73e5-5fc8-4d84-a5ac-3357ec94fa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:13.623565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1301/2605] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:13.631404 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f2110-d189-4a60-8406-90ead4578d6e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.631688 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f2110-d189-4a60-8406-90ead4578d6e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.631962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f2110-d189-4a60-8406-90ead4578d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:13.657878 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3d4f2110-d189-4a60-8406-90ead4578d6e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:13.659140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3d4f2110-d189-4a60-8406-90ead4578d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:13.669324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1305/2606] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:13.674385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d73e5-5fc8-4d84-a5ac-3357ec94fa52 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:13.685265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d73e5-5fc8-4d84-a5ac-3357ec94fa52 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:13.690304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb6ebd-931c-43ee-9c5d-060efadec2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:13.690971 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb6ebd-931c-43ee-9c5d-060efadec2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.691346 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb6ebd-931c-43ee-9c5d-060efadec2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.691694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb6ebd-931c-43ee-9c5d-060efadec2c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:13.692407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb6ebd-931c-43ee-9c5d-060efadec2c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:13.692773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb6ebd-931c-43ee-9c5d-060efadec2c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:13.694464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb6ebd-931c-43ee-9c5d-060efadec2c9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:13.706450 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1302/2607] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:35:13.710698 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f2110-d189-4a60-8406-90ead4578d6e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:13.719856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f2110-d189-4a60-8406-90ead4578d6e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:13.724745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2748b38-2b8e-497c-a63f-8bd1a4db6b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:13.741829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2748b38-2b8e-497c-a63f-8bd1a4db6b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.742163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2748b38-2b8e-497c-a63f-8bd1a4db6b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.742444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2748b38-2b8e-497c-a63f-8bd1a4db6b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:13.743785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2748b38-2b8e-497c-a63f-8bd1a4db6b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:13.746907 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2748b38-2b8e-497c-a63f-8bd1a4db6b9a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:13.782747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1306/2608] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:13 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:13.786925 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb6ebd-931c-43ee-9c5d-060efadec2c9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:13.798200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb6ebd-931c-43ee-9c5d-060efadec2c9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:13.859206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c274e4-d720-4eef-968a-f228b4b16cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:13.859819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c274e4-d720-4eef-968a-f228b4b16cbc admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.860101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c274e4-d720-4eef-968a-f228b4b16cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.860379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c274e4-d720-4eef-968a-f228b4b16cbc admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:13.861293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c274e4-d720-4eef-968a-f228b4b16cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:13.863253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c274e4-d720-4eef-968a-f228b4b16cbc admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:13.889220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1303/2609] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:13 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:13.890476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2748b38-2b8e-497c-a63f-8bd1a4db6b9a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:13.901675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2748b38-2b8e-497c-a63f-8bd1a4db6b9a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:13.907586 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb287-597f-4171-84e5-956a29aecfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:13.908330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb287-597f-4171-84e5-956a29aecfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.908908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb287-597f-4171-84e5-956a29aecfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.909046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb287-597f-4171-84e5-956a29aecfc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:13.909717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cb287-597f-4171-84e5-956a29aecfc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:13.911740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cb287-597f-4171-84e5-956a29aecfc7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:13.926083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1307/2610] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:13.929281 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c274e4-d720-4eef-968a-f228b4b16cbc admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:13.939831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c274e4-d720-4eef-968a-f228b4b16cbc admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:13.944633 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af239d-64a2-4c15-b19e-28c7c723499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:13.945245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af239d-64a2-4c15-b19e-28c7c723499a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.945523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af239d-64a2-4c15-b19e-28c7c723499a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.945850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af239d-64a2-4c15-b19e-28c7c723499a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:13.946700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af239d-64a2-4c15-b19e-28c7c723499a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:13.948345 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af239d-64a2-4c15-b19e-28c7c723499a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:13.953896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:13.968005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1304/2611] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:13.968776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb287-597f-4171-84e5-956a29aecfc7 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:13.980934 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb287-597f-4171-84e5-956a29aecfc7 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:13.985652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811181a8-d178-4459-825e-6ca34724bf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:13.986686 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811181a8-d178-4459-825e-6ca34724bf5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:13.986965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811181a8-d178-4459-825e-6ca34724bf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:13.987241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811181a8-d178-4459-825e-6ca34724bf5c admin admin] PATH_INFO: `/v3/projects/e9e79de9126e4dcb91843e386b377806/users/f7d5593c0de84dbaba77dd8ae0ed425c/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.005965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811181a8-d178-4459-825e-6ca34724bf5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e79de9126e4dcb91843e386b377806, user_id=f7d5593c0de84dbaba77dd8ae0ed425c, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:14.008914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811181a8-d178-4459-825e-6ca34724bf5c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:14.026288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1308/2612] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:13 2022] PUT /identity/v3/projects/e9e79de9126e4dcb91843e386b377806/users/f7d5593c0de84dbaba77dd8ae0ed425c/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:14.037923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b4c8f-85c7-470d-afe3-636979b72db3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:14.038234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b4c8f-85c7-470d-afe3-636979b72db3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:14.038508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b4c8f-85c7-470d-afe3-636979b72db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.072306 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e57b4c8f-85c7-470d-afe3-636979b72db3 None None] MFA Rules not processed for user `f7d5593c0de84dbaba77dd8ae0ed425c`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:14.073632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e57b4c8f-85c7-470d-afe3-636979b72db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:14.162769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811181a8-d178-4459-825e-6ca34724bf5c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:14.162969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1305/2613] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:14.259956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811181a8-d178-4459-825e-6ca34724bf5c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:14.265805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24209731-a714-442b-be7a-249f46f33ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:14.267422 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24209731-a714-442b-be7a-249f46f33ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:14.267895 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24209731-a714-442b-be7a-249f46f33ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:14.268494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24209731-a714-442b-be7a-249f46f33ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.269504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24209731-a714-442b-be7a-249f46f33ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:14.280168 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24209731-a714-442b-be7a-249f46f33ec9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:14.303148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:14.334442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1309/2614] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:14 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:14.339156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b4c8f-85c7-470d-afe3-636979b72db3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:14.358274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b4c8f-85c7-470d-afe3-636979b72db3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:14.469516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1d21b-d89b-42d5-b14b-33e165f03213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:14.470577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1d21b-d89b-42d5-b14b-33e165f03213 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:14.471092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1d21b-d89b-42d5-b14b-33e165f03213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:14.472835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1d21b-d89b-42d5-b14b-33e165f03213 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.474472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1d21b-d89b-42d5-b14b-33e165f03213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:14.478479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1d21b-d89b-42d5-b14b-33e165f03213 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:14.510622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1306/2615] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:14 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:14.514827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24209731-a714-442b-be7a-249f46f33ec9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:14.532822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24209731-a714-442b-be7a-249f46f33ec9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:14.544132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30bf3f-2689-4587-a1c9-575ab297c03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:14.544132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30bf3f-2689-4587-a1c9-575ab297c03b admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:14.545271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30bf3f-2689-4587-a1c9-575ab297c03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:14.545271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30bf3f-2689-4587-a1c9-575ab297c03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.546117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30bf3f-2689-4587-a1c9-575ab297c03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:14.548078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30bf3f-2689-4587-a1c9-575ab297c03b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:14.559382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1310/2616] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:35:14.563444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1d21b-d89b-42d5-b14b-33e165f03213 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:14.575225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1d21b-d89b-42d5-b14b-33e165f03213 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:14.580056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a784b-a626-4e04-bbbb-d3758a651dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:14.580792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a784b-a626-4e04-bbbb-d3758a651dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:14.581070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a784b-a626-4e04-bbbb-d3758a651dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:14.581368 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a784b-a626-4e04-bbbb-d3758a651dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.582182 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a784b-a626-4e04-bbbb-d3758a651dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:14.583875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a784b-a626-4e04-bbbb-d3758a651dc3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:14.599616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1307/2617] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:14.600175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30bf3f-2689-4587-a1c9-575ab297c03b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:14.610298 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30bf3f-2689-4587-a1c9-575ab297c03b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:14.616808 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2a3bc-7e36-47bf-a323-b82a9c6772a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:14.617676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2a3bc-7e36-47bf-a323-b82a9c6772a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:14.618588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2a3bc-7e36-47bf-a323-b82a9c6772a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:14.618940 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2a3bc-7e36-47bf-a323-b82a9c6772a9 admin admin] PATH_INFO: `/v3/projects/6732b91fb4634660b12b4afc456fe581/users/868079f7422e42baa754bf3fd1542329/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.637155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2a3bc-7e36-47bf-a323-b82a9c6772a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6732b91fb4634660b12b4afc456fe581, user_id=868079f7422e42baa754bf3fd1542329, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:14.640007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2a3bc-7e36-47bf-a323-b82a9c6772a9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:14.659342 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1311/2618] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:14 2022] PUT /identity/v3/projects/6732b91fb4634660b12b4afc456fe581/users/868079f7422e42baa754bf3fd1542329/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:14.670774 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9080ed-9fa1-4ecd-9da3-af05e4fd46eb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:14.671064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9080ed-9fa1-4ecd-9da3-af05e4fd46eb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:14.671328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9080ed-9fa1-4ecd-9da3-af05e4fd46eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.704031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8c9080ed-9fa1-4ecd-9da3-af05e4fd46eb None None] MFA Rules not processed for user `868079f7422e42baa754bf3fd1542329`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:14.705259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8c9080ed-9fa1-4ecd-9da3-af05e4fd46eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:14.784231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:14.792043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1308/2619] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:14.796288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2a3bc-7e36-47bf-a323-b82a9c6772a9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:14.861460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2a3bc-7e36-47bf-a323-b82a9c6772a9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:14.866340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e7d49-90f4-4e01-946b-3f57b04fb861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:14.866941 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e7d49-90f4-4e01-946b-3f57b04fb861 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:14.867218 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e7d49-90f4-4e01-946b-3f57b04fb861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:14.867489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e7d49-90f4-4e01-946b-3f57b04fb861 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.868175 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e7d49-90f4-4e01-946b-3f57b04fb861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:14.870054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e7d49-90f4-4e01-946b-3f57b04fb861 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:14.895212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1312/2620] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:14 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:14.899967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9080ed-9fa1-4ecd-9da3-af05e4fd46eb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:14.910805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9080ed-9fa1-4ecd-9da3-af05e4fd46eb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:14.970245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44472f3b-748d-4ba1-95b3-6d357fabd1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:14.970952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44472f3b-748d-4ba1-95b3-6d357fabd1ea admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:14.971127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44472f3b-748d-4ba1-95b3-6d357fabd1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:14.971396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44472f3b-748d-4ba1-95b3-6d357fabd1ea admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:14.972038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44472f3b-748d-4ba1-95b3-6d357fabd1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:14.973974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44472f3b-748d-4ba1-95b3-6d357fabd1ea admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:14.997422 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1309/2621] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:14 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:15.001427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e7d49-90f4-4e01-946b-3f57b04fb861 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:15.012650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e7d49-90f4-4e01-946b-3f57b04fb861 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:15.017613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2f79d-e5fd-42fd-95c9-8c5f1091b563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:15.018373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2f79d-e5fd-42fd-95c9-8c5f1091b563 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:15.018801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2f79d-e5fd-42fd-95c9-8c5f1091b563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:15.019114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2f79d-e5fd-42fd-95c9-8c5f1091b563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:15.019933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2f79d-e5fd-42fd-95c9-8c5f1091b563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:15.021636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2f79d-e5fd-42fd-95c9-8c5f1091b563 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:15.033530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1313/2622] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:35:15.038038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44472f3b-748d-4ba1-95b3-6d357fabd1ea admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:15.049026 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44472f3b-748d-4ba1-95b3-6d357fabd1ea admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:15.054092 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969c2a2-6797-4b54-979f-e4da85199db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:15.054755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969c2a2-6797-4b54-979f-e4da85199db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:15.055031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969c2a2-6797-4b54-979f-e4da85199db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:15.055328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969c2a2-6797-4b54-979f-e4da85199db2 admin admin] PATH_INFO: `/v3/projects/749b305594234d2a8c18a1facb6161e1/users/73f9b1260a094e79afb4f12bccc2d0f3/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:15.070111 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969c2a2-6797-4b54-979f-e4da85199db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749b305594234d2a8c18a1facb6161e1, user_id=73f9b1260a094e79afb4f12bccc2d0f3, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:15.073328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969c2a2-6797-4b54-979f-e4da85199db2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:15.089766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1310/2623] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:15 2022] PUT /identity/v3/projects/749b305594234d2a8c18a1facb6161e1/users/73f9b1260a094e79afb4f12bccc2d0f3/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:15.093702 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2f79d-e5fd-42fd-95c9-8c5f1091b563 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:15.103775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2f79d-e5fd-42fd-95c9-8c5f1091b563 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:15.165556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc60b1-1469-4b77-ae69-f6a5207bbb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:15.166299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc60b1-1469-4b77-ae69-f6a5207bbb85 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:15.166577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc60b1-1469-4b77-ae69-f6a5207bbb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:15.166872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc60b1-1469-4b77-ae69-f6a5207bbb85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:15.167501 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc60b1-1469-4b77-ae69-f6a5207bbb85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:15.169078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc60b1-1469-4b77-ae69-f6a5207bbb85 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:15.179055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1314/2624] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:15.183362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969c2a2-6797-4b54-979f-e4da85199db2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:15.193687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969c2a2-6797-4b54-979f-e4da85199db2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:15.198830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747fd71-71ea-4c2a-980b-9dad20ee0b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:15.199471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747fd71-71ea-4c2a-980b-9dad20ee0b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:15.199747 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747fd71-71ea-4c2a-980b-9dad20ee0b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:15.200016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747fd71-71ea-4c2a-980b-9dad20ee0b97 admin admin] PATH_INFO: `/v3/projects/749b305594234d2a8c18a1facb6161e1/users/73f9b1260a094e79afb4f12bccc2d0f3/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:15.201879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747fd71-71ea-4c2a-980b-9dad20ee0b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749b305594234d2a8c18a1facb6161e1, user_id=73f9b1260a094e79afb4f12bccc2d0f3, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:15.204504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747fd71-71ea-4c2a-980b-9dad20ee0b97 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:15.215527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1311/2625] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:15 2022] PUT /identity/v3/projects/749b305594234d2a8c18a1facb6161e1/users/73f9b1260a094e79afb4f12bccc2d0f3/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:15.225753 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeec3f-0f06-4f76-ade1-fcec43497980 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:15.226080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeec3f-0f06-4f76-ade1-fcec43497980 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:15.226360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeec3f-0f06-4f76-ade1-fcec43497980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:15.253584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6bfeec3f-0f06-4f76-ade1-fcec43497980 None None] MFA Rules not processed for user `73f9b1260a094e79afb4f12bccc2d0f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:15.254845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6bfeec3f-0f06-4f76-ade1-fcec43497980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:15.267745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:15.322759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1315/2626] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:15.333803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747fd71-71ea-4c2a-980b-9dad20ee0b97 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:15.335507 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f747fd71-71ea-4c2a-980b-9dad20ee0b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:15.395545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747fd71-71ea-4c2a-980b-9dad20ee0b97 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:15.399921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81dbce-eefa-448b-b848-8f0e0408251a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:15.400515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81dbce-eefa-448b-b848-8f0e0408251a service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:15.400789 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81dbce-eefa-448b-b848-8f0e0408251a service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:15.401056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81dbce-eefa-448b-b848-8f0e0408251a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:15.410343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81dbce-eefa-448b-b848-8f0e0408251a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:15.411757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81dbce-eefa-448b-b848-8f0e0408251a service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:15.421386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1312/2627] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:15.935408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeec3f-0f06-4f76-ade1-fcec43497980 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:15.944780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeec3f-0f06-4f76-ade1-fcec43497980 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:15.948278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b97d96-92d7-4708-a85b-4c438e6b7704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:15.948872 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b97d96-92d7-4708-a85b-4c438e6b7704 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:15.949146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b97d96-92d7-4708-a85b-4c438e6b7704 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:15.949416 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b97d96-92d7-4708-a85b-4c438e6b7704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:16.016183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b97d96-92d7-4708-a85b-4c438e6b7704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:16.018144 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b97d96-92d7-4708-a85b-4c438e6b7704 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:16.028482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1316/2628] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:16.521617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81dbce-eefa-448b-b848-8f0e0408251a service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:16.523162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9f81dbce-eefa-448b-b848-8f0e0408251a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:16.573746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:16.587582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81dbce-eefa-448b-b848-8f0e0408251a service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:16.592566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:16.593199 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:16.593479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:16.593782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] PATH_INFO: `/v3/users/53cc6ddee574442486f78d9ed033cd2e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:16.594906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53cc6ddee574442486f78d9ed033cd2e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:16.596934 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:16.644195 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53cc6ddee574442486f78d9ed033cd2e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:16.644757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53cc6ddee574442486f78d9ed033cd2e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:16.651892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53cc6ddee574442486f78d9ed033cd2e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:16.652689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53cc6ddee574442486f78d9ed033cd2e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:16.659735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53cc6ddee574442486f78d9ed033cd2e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:16.677731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1313/2629] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:16 2022] DELETE /identity/v3/users/53cc6ddee574442486f78d9ed033cd2e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:16.682877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe1707-46c6-4bcd-ae3b-bdefba8a488e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:16.683169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe1707-46c6-4bcd-ae3b-bdefba8a488e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:16.683445 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe1707-46c6-4bcd-ae3b-bdefba8a488e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:16.702090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-58fe1707-46c6-4bcd-ae3b-bdefba8a488e None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:16.703300 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-58fe1707-46c6-4bcd-ae3b-bdefba8a488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:16.760766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1317/2630] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:16.770393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:16.771842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:16.828097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df6d7c-bc58-408a-8f51-a9504be71110 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:16.831707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f8f0-2c0a-47d8-bbf9-9b6886012aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:16.832299 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f8f0-2c0a-47d8-bbf9-9b6886012aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:16.832574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f8f0-2c0a-47d8-bbf9-9b6886012aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:16.832843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f8f0-2c0a-47d8-bbf9-9b6886012aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:16.842027 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42f8f0-2c0a-47d8-bbf9-9b6886012aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:16.843443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42f8f0-2c0a-47d8-bbf9-9b6886012aa7 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:16.852997 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1314/2631] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:17.120207 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe1707-46c6-4bcd-ae3b-bdefba8a488e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:17.122052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-58fe1707-46c6-4bcd-ae3b-bdefba8a488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:17.138326 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe1707-46c6-4bcd-ae3b-bdefba8a488e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:17.148163 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:17.148812 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:17.149624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:17.149888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] PATH_INFO: `/v3/projects/9e967d18b1214844af2c4a7a1bd22c67` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:17.151206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e967d18b1214844af2c4a7a1bd22c67)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:17.153129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:17.199577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:17.202783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e967d18b1214844af2c4a7a1bd22c67', 'request_id': 'req-9765c788-83ff-437f-8cdd-49926d52bcff'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:17.212133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e967d18b1214844af2c4a7a1bd22c67', 'request_id': 'req-9765c788-83ff-437f-8cdd-49926d52bcff'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:17.218847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e967d18b1214844af2c4a7a1bd22c67', 'request_id': 'req-9765c788-83ff-437f-8cdd-49926d52bcff'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:17.232306 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] The token cache is being invalidate because project 9e967d18b1214844af2c4a7a1bd22c67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:17.232734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:17.233420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:17.234932 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1318/2632] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:17 2022] DELETE /identity/v3/projects/9e967d18b1214844af2c4a7a1bd22c67 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:20.109926 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f8f0-2c0a-47d8-bbf9-9b6886012aa7 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:20.111677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ad42f8f0-2c0a-47d8-bbf9-9b6886012aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:20.194409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:20.197522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f8f0-2c0a-47d8-bbf9-9b6886012aa7 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:20.246298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e296f6-3594-4070-a231-3f7a7870d842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:20.246896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e296f6-3594-4070-a231-3f7a7870d842 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:20.247171 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e296f6-3594-4070-a231-3f7a7870d842 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:20.247440 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e296f6-3594-4070-a231-3f7a7870d842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:20.249136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-50e296f6-3594-4070-a231-3f7a7870d842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:20.318404 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e296f6-3594-4070-a231-3f7a7870d842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:20.319827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e296f6-3594-4070-a231-3f7a7870d842 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:20.329033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1315/2633] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:22.403103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:22.404513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:22.526667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765c788-83ff-437f-8cdd-49926d52bcff admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:22.531229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:22.531850 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:22.532125 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:22.532396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] PATH_INFO: `/v3/users/f7d5593c0de84dbaba77dd8ae0ed425c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:22.533609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7d5593c0de84dbaba77dd8ae0ed425c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:22.535635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:22.581259 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7d5593c0de84dbaba77dd8ae0ed425c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:22.588464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7d5593c0de84dbaba77dd8ae0ed425c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:22.589174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7d5593c0de84dbaba77dd8ae0ed425c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:22.594397 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7d5593c0de84dbaba77dd8ae0ed425c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:22.601646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7d5593c0de84dbaba77dd8ae0ed425c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:22.615414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:22.621152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1319/2634] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:22 2022] DELETE /identity/v3/users/f7d5593c0de84dbaba77dd8ae0ed425c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:22.625928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a2bb7-5c67-4058-b8fb-20ce152a1e99 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:22.626436 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a2bb7-5c67-4058-b8fb-20ce152a1e99 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:22.626552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a2bb7-5c67-4058-b8fb-20ce152a1e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:22.653296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7e6a2bb7-5c67-4058-b8fb-20ce152a1e99 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:22.654586 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7e6a2bb7-5c67-4058-b8fb-20ce152a1e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:22.710927 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1316/2635] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:22.720638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:22.722047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:22.779034 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515f7d-0021-4c8e-bc04-aec8b602002b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:22.782641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e6b81-bad4-439f-be30-3d2d28855854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:22.783236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6b81-bad4-439f-be30-3d2d28855854 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:22.783515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6b81-bad4-439f-be30-3d2d28855854 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:22.783784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6b81-bad4-439f-be30-3d2d28855854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:22.792960 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e6b81-bad4-439f-be30-3d2d28855854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:22.794393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e6b81-bad4-439f-be30-3d2d28855854 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:22.803479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1320/2636] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:22.922990 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a2bb7-5c67-4058-b8fb-20ce152a1e99 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:22.924517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7e6a2bb7-5c67-4058-b8fb-20ce152a1e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:22.936765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a2bb7-5c67-4058-b8fb-20ce152a1e99 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:22.941634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:22.942380 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:22.942728 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:22.943067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] PATH_INFO: `/v3/users/d388f9040af147229dbeb3e46353b49d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:22.944955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d388f9040af147229dbeb3e46353b49d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:22.946696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:22.983048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:22.996453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd388f9040af147229dbeb3e46353b49d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:22.997143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd388f9040af147229dbeb3e46353b49d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.003978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd388f9040af147229dbeb3e46353b49d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.004751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd388f9040af147229dbeb3e46353b49d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.012064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd388f9040af147229dbeb3e46353b49d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.034411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1317/2637] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:22 2022] DELETE /identity/v3/users/d388f9040af147229dbeb3e46353b49d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:23.038997 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d385f-657f-437e-82c5-03d2e2ac108d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:23.039285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d385f-657f-437e-82c5-03d2e2ac108d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:23.039603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d385f-657f-437e-82c5-03d2e2ac108d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:23.059318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2f6d385f-657f-437e-82c5-03d2e2ac108d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:23.060544 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2f6d385f-657f-437e-82c5-03d2e2ac108d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:23.076115 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:23.077592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:23.128561 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1321/2638] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:23.138450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d385f-657f-437e-82c5-03d2e2ac108d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:23.139777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2f6d385f-657f-437e-82c5-03d2e2ac108d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:23.150107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d16e2-5538-4307-80ca-79f048667ea4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:23.154951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:23.156382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:23.156662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:23.156938 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] PATH_INFO: `/v3/projects/e9e79de9126e4dcb91843e386b377806` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:23.158145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e79de9126e4dcb91843e386b377806)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:23.160083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:23.198098 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d385f-657f-437e-82c5-03d2e2ac108d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:23.206456 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e79de9126e4dcb91843e386b377806', 'request_id': 'req-eb31dc07-5558-4c0b-99cb-88c6029f3419'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.212528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e79de9126e4dcb91843e386b377806', 'request_id': 'req-eb31dc07-5558-4c0b-99cb-88c6029f3419'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.220020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e79de9126e4dcb91843e386b377806', 'request_id': 'req-eb31dc07-5558-4c0b-99cb-88c6029f3419'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.233429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] The token cache is being invalidate because project e9e79de9126e4dcb91843e386b377806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:23.233797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.234529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.236382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1318/2639] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:23 2022] DELETE /identity/v3/projects/e9e79de9126e4dcb91843e386b377806 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:23.240202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:23.241500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:23.248694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:23.249305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:23.249670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:23.249973 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:23.251709 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:23.264941 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:23.285052 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:23.307439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31dc07-5558-4c0b-99cb-88c6029f3419 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:23.311976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:23.312603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:23.312880 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:23.313148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] PATH_INFO: `/v3/users/868079f7422e42baa754bf3fd1542329` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:23.314300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=868079f7422e42baa754bf3fd1542329)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:23.315651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:23.316223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:23.317072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:23.332717 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1322/2640] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:35:23.374062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868079f7422e42baa754bf3fd1542329'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.375107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868079f7422e42baa754bf3fd1542329'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.384360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868079f7422e42baa754bf3fd1542329'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.385008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868079f7422e42baa754bf3fd1542329'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.392371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868079f7422e42baa754bf3fd1542329'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.413960 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1319/2641] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:23 2022] DELETE /identity/v3/users/868079f7422e42baa754bf3fd1542329 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:23.616068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:23.617760 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:23.630599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50657d-2f27-40b2-a2d4-cdde4c6953df service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:23.692565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:23.693214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:23.693489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:23.693818 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] PATH_INFO: `/v3/projects/20e18203e3fe44feab8f9afbba026f3d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:23.695270 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e18203e3fe44feab8f9afbba026f3d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:23.696049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:23.697240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:23.698587 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:23.715550 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffc1e3-dcbd-4933-9302-1989a5ccd568 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:23.720538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:23.721183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:23.721464 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:23.721735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] PATH_INFO: `/v3/projects/6732b91fb4634660b12b4afc456fe581` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:23.723492 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6732b91fb4634660b12b4afc456fe581)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:23.725513 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:23.753749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e18203e3fe44feab8f9afbba026f3d', 'request_id': 'req-9e8b2764-eadf-46f2-a76e-818866d89bbb'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.760769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e18203e3fe44feab8f9afbba026f3d', 'request_id': 'req-9e8b2764-eadf-46f2-a76e-818866d89bbb'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.767591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e18203e3fe44feab8f9afbba026f3d', 'request_id': 'req-9e8b2764-eadf-46f2-a76e-818866d89bbb'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.778311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6732b91fb4634660b12b4afc456fe581', 'request_id': 'req-4686d507-cfeb-4e5a-ac05-904d1017407d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.779417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] The token cache is being invalidate because project 20e18203e3fe44feab8f9afbba026f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:23.779708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.780390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.781865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1323/2642] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:23 2022] DELETE /identity/v3/projects/20e18203e3fe44feab8f9afbba026f3d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:23.791713 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6732b91fb4634660b12b4afc456fe581', 'request_id': 'req-4686d507-cfeb-4e5a-ac05-904d1017407d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.808353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6732b91fb4634660b12b4afc456fe581', 'request_id': 'req-4686d507-cfeb-4e5a-ac05-904d1017407d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.821382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] The token cache is being invalidate because project 6732b91fb4634660b12b4afc456fe581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:23.821704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.822454 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:23.824206 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1320/2643] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:23 2022] DELETE /identity/v3/projects/6732b91fb4634660b12b4afc456fe581 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:23.828149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:23.829598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:23.877217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:23.902582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b2764-eadf-46f2-a76e-818866d89bbb admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:23.963526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:23.964096 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:23.964380 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:23.964652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] PATH_INFO: `/v3/users/73f9b1260a094e79afb4f12bccc2d0f3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:23.965834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f9b1260a094e79afb4f12bccc2d0f3)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:23.967819 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:24.021601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f9b1260a094e79afb4f12bccc2d0f3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.030481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f9b1260a094e79afb4f12bccc2d0f3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.031240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f9b1260a094e79afb4f12bccc2d0f3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.032054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f9b1260a094e79afb4f12bccc2d0f3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.039621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb455e97-f25c-4c84-94e3-37526080d9f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f9b1260a094e79afb4f12bccc2d0f3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.056980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1324/2644] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:23 2022] DELETE /identity/v3/users/73f9b1260a094e79afb4f12bccc2d0f3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:24.327231 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:24.329162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:24.344279 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686d507-cfeb-4e5a-ac05-904d1017407d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:24.389833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:24.407756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:24.408386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.408664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.408937 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] PATH_INFO: `/v3/projects/749b305594234d2a8c18a1facb6161e1` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.410075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=749b305594234d2a8c18a1facb6161e1)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:24.412000 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:24.456780 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749b305594234d2a8c18a1facb6161e1', 'request_id': 'req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.462288 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749b305594234d2a8c18a1facb6161e1', 'request_id': 'req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.469987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749b305594234d2a8c18a1facb6161e1', 'request_id': 'req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.480787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] The token cache is being invalidate because project 749b305594234d2a8c18a1facb6161e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:24.481073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.481697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d30263c9-7c5c-478d-ac3d-b8dd1c8ce9f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:24.483172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1321/2645] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:24 2022] DELETE /identity/v3/projects/749b305594234d2a8c18a1facb6161e1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:24.489497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb93513-4e7d-4546-845f-cc9edb75b667 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.489732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb93513-4e7d-4546-845f-cc9edb75b667 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.490046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb93513-4e7d-4546-845f-cc9edb75b667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.539761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-bdb93513-4e7d-4546-845f-cc9edb75b667 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:24.540963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bdb93513-4e7d-4546-845f-cc9edb75b667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:24.585295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:24.607952 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1325/2646] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:24.616479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c8112-1045-4fd8-b1aa-3ea55a2318ae None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.616766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c8112-1045-4fd8-b1aa-3ea55a2318ae None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.617038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c8112-1045-4fd8-b1aa-3ea55a2318ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.637373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6e8c8112-1045-4fd8-b1aa-3ea55a2318ae None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:24.638623 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6e8c8112-1045-4fd8-b1aa-3ea55a2318ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:24.646378 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1322/2647] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:24.650566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb93513-4e7d-4546-845f-cc9edb75b667 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:24.660311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb93513-4e7d-4546-845f-cc9edb75b667 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:24.664810 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30a47d-d4f8-4eec-bdd0-e0035d6944f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:24.665403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30a47d-d4f8-4eec-bdd0-e0035d6944f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.665683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30a47d-d4f8-4eec-bdd0-e0035d6944f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.665983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30a47d-d4f8-4eec-bdd0-e0035d6944f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.666625 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30a47d-d4f8-4eec-bdd0-e0035d6944f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:24.666929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30a47d-d4f8-4eec-bdd0-e0035d6944f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:24.668522 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30a47d-d4f8-4eec-bdd0-e0035d6944f6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:24.679815 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1326/2648] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:24.684563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c8112-1045-4fd8-b1aa-3ea55a2318ae None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:24.693498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c8112-1045-4fd8-b1aa-3ea55a2318ae None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:24.699007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3dd66-e7ac-4c4e-b1b4-82b84c572d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:24.699602 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3dd66-e7ac-4c4e-b1b4-82b84c572d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.699879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3dd66-e7ac-4c4e-b1b4-82b84c572d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.700150 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3dd66-e7ac-4c4e-b1b4-82b84c572d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.700764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3dd66-e7ac-4c4e-b1b4-82b84c572d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:24.702675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3dd66-e7ac-4c4e-b1b4-82b84c572d2f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:24.730490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1323/2649] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:24 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:24.734643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30a47d-d4f8-4eec-bdd0-e0035d6944f6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:24.746899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30a47d-d4f8-4eec-bdd0-e0035d6944f6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:24.802349 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b175d0-2a9c-46fd-847e-5831b367c395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:24.802951 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b175d0-2a9c-46fd-847e-5831b367c395 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.803233 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b175d0-2a9c-46fd-847e-5831b367c395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.803504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b175d0-2a9c-46fd-847e-5831b367c395 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.804138 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b175d0-2a9c-46fd-847e-5831b367c395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:24.806937 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b175d0-2a9c-46fd-847e-5831b367c395 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:24.827558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1327/2650] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:24 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:24.831634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3dd66-e7ac-4c4e-b1b4-82b84c572d2f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:24.841708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3dd66-e7ac-4c4e-b1b4-82b84c572d2f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:24.846383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c174b9-6458-4c2c-a1f3-5de264c79169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:24.847007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c174b9-6458-4c2c-a1f3-5de264c79169 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.847291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c174b9-6458-4c2c-a1f3-5de264c79169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.847589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c174b9-6458-4c2c-a1f3-5de264c79169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.848226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c174b9-6458-4c2c-a1f3-5de264c79169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:24.849829 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c174b9-6458-4c2c-a1f3-5de264c79169 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:24.860178 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1324/2651] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:24.864237 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b175d0-2a9c-46fd-847e-5831b367c395 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:24.874902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b175d0-2a9c-46fd-847e-5831b367c395 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:24.879547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb29b6f-8fc6-4b71-80d4-b155a16a2f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:24.880135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb29b6f-8fc6-4b71-80d4-b155a16a2f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.880414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb29b6f-8fc6-4b71-80d4-b155a16a2f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.880689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb29b6f-8fc6-4b71-80d4-b155a16a2f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.881322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb29b6f-8fc6-4b71-80d4-b155a16a2f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:24.882921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb29b6f-8fc6-4b71-80d4-b155a16a2f92 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:24.893240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1328/2652] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:24.897100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c174b9-6458-4c2c-a1f3-5de264c79169 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:24.906700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c174b9-6458-4c2c-a1f3-5de264c79169 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:24.911219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6557f-7c1c-47a0-b469-0620729a0e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:24.911852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6557f-7c1c-47a0-b469-0620729a0e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.912130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6557f-7c1c-47a0-b469-0620729a0e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.912401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6557f-7c1c-47a0-b469-0620729a0e36 admin admin] PATH_INFO: `/v3/projects/39cfd41d37c948c183b7586b27bd4189/users/7ecb1b338f17425e917b4977e0f1b27c/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.926928 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6557f-7c1c-47a0-b469-0620729a0e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39cfd41d37c948c183b7586b27bd4189, user_id=7ecb1b338f17425e917b4977e0f1b27c, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:24.929555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6557f-7c1c-47a0-b469-0620729a0e36 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:24.942500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1325/2653] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:24 2022] PUT /identity/v3/projects/39cfd41d37c948c183b7586b27bd4189/users/7ecb1b338f17425e917b4977e0f1b27c/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:24.953504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ba2c-04d3-4135-9853-5f86dc193f11 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:24.953788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ba2c-04d3-4135-9853-5f86dc193f11 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:24.954106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ba2c-04d3-4135-9853-5f86dc193f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:24.981631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-af64ba2c-04d3-4135-9853-5f86dc193f11 None None] MFA Rules not processed for user `7ecb1b338f17425e917b4977e0f1b27c`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:24.982880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-af64ba2c-04d3-4135-9853-5f86dc193f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:25.049146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1329/2654] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:25.053618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6557f-7c1c-47a0-b469-0620729a0e36 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:25.115364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6557f-7c1c-47a0-b469-0620729a0e36 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:25.119998 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1313de-7e7a-4a1b-9220-d4dd270d8859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:25.120589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1313de-7e7a-4a1b-9220-d4dd270d8859 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.120864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1313de-7e7a-4a1b-9220-d4dd270d8859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.121134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1313de-7e7a-4a1b-9220-d4dd270d8859 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:25.121763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1313de-7e7a-4a1b-9220-d4dd270d8859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:25.123657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1313de-7e7a-4a1b-9220-d4dd270d8859 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:25.137695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:25.150950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1326/2655] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:25 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:25.155018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64ba2c-04d3-4135-9853-5f86dc193f11 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:25.166006 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64ba2c-04d3-4135-9853-5f86dc193f11 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:25.199446 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:25.223594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92545590-7065-497b-b076-7c4afdf9be68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:25.224199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92545590-7065-497b-b076-7c4afdf9be68 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.224481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92545590-7065-497b-b076-7c4afdf9be68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.224757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92545590-7065-497b-b076-7c4afdf9be68 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:25.225388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92545590-7065-497b-b076-7c4afdf9be68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:25.227964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92545590-7065-497b-b076-7c4afdf9be68 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:25.248178 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1330/2656] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:25 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:25.252187 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1313de-7e7a-4a1b-9220-d4dd270d8859 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:25.262488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1313de-7e7a-4a1b-9220-d4dd270d8859 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:25.267376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527ed82-c126-405d-a19f-0d0a301670cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:25.267978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527ed82-c126-405d-a19f-0d0a301670cf admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.268257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527ed82-c126-405d-a19f-0d0a301670cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.268529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527ed82-c126-405d-a19f-0d0a301670cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:25.269160 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527ed82-c126-405d-a19f-0d0a301670cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:25.270783 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527ed82-c126-405d-a19f-0d0a301670cf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:25.281615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1327/2657] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:25.285588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92545590-7065-497b-b076-7c4afdf9be68 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:25.297119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92545590-7065-497b-b076-7c4afdf9be68 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:25.302563 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d1210-3ae8-4f4c-a8db-60260d9e05c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:25.302863 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d1210-3ae8-4f4c-a8db-60260d9e05c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.303136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d1210-3ae8-4f4c-a8db-60260d9e05c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.303405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d1210-3ae8-4f4c-a8db-60260d9e05c3 admin admin] PATH_INFO: `/v3/projects/4ed7d8a2107d45e6836d8ad52a753a6f/users/059465771c3f49638074033455b12e1f/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:25.318174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d1210-3ae8-4f4c-a8db-60260d9e05c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed7d8a2107d45e6836d8ad52a753a6f, user_id=059465771c3f49638074033455b12e1f, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:25.320845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d1210-3ae8-4f4c-a8db-60260d9e05c3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:25.334323 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1331/2658] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:25 2022] PUT /identity/v3/projects/4ed7d8a2107d45e6836d8ad52a753a6f/users/059465771c3f49638074033455b12e1f/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:25.338956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527ed82-c126-405d-a19f-0d0a301670cf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:25.349081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527ed82-c126-405d-a19f-0d0a301670cf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:25.413224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc86bce-464c-4ae4-9deb-e177efdc809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:25.413224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86bce-464c-4ae4-9deb-e177efdc809e admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.413224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86bce-464c-4ae4-9deb-e177efdc809e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.413224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86bce-464c-4ae4-9deb-e177efdc809e admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:25.413756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc86bce-464c-4ae4-9deb-e177efdc809e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:25.413756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc86bce-464c-4ae4-9deb-e177efdc809e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:25.424316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1328/2659] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:25.428401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d1210-3ae8-4f4c-a8db-60260d9e05c3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:25.438670 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d1210-3ae8-4f4c-a8db-60260d9e05c3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:25.443325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fe652-08df-4303-8b80-9c3f87e6abc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:25.443975 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fe652-08df-4303-8b80-9c3f87e6abc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.444267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fe652-08df-4303-8b80-9c3f87e6abc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.444589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fe652-08df-4303-8b80-9c3f87e6abc4 admin admin] PATH_INFO: `/v3/projects/4ed7d8a2107d45e6836d8ad52a753a6f/users/059465771c3f49638074033455b12e1f/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:25.446623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fe652-08df-4303-8b80-9c3f87e6abc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed7d8a2107d45e6836d8ad52a753a6f, user_id=059465771c3f49638074033455b12e1f, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:25.449135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fe652-08df-4303-8b80-9c3f87e6abc4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:25.459534 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1332/2660] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:25 2022] PUT /identity/v3/projects/4ed7d8a2107d45e6836d8ad52a753a6f/users/059465771c3f49638074033455b12e1f/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:25.471582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3588422-3d2e-4b8a-a552-181efea0a98a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.471858 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3588422-3d2e-4b8a-a552-181efea0a98a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.472158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3588422-3d2e-4b8a-a552-181efea0a98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:25.500753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d3588422-3d2e-4b8a-a552-181efea0a98a None None] MFA Rules not processed for user `059465771c3f49638074033455b12e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:25.502007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d3588422-3d2e-4b8a-a552-181efea0a98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:25.579913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1329/2661] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:25 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:25.595885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fe652-08df-4303-8b80-9c3f87e6abc4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:25.598018 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f65fe652-08df-4303-8b80-9c3f87e6abc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:25.660536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fe652-08df-4303-8b80-9c3f87e6abc4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:25.664910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e5b5c-c567-4a8b-9725-3f955a7e9ec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:25.665541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5b5c-c567-4a8b-9725-3f955a7e9ec7 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.665821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5b5c-c567-4a8b-9725-3f955a7e9ec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.666126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5b5c-c567-4a8b-9725-3f955a7e9ec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:25.675605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e5b5c-c567-4a8b-9725-3f955a7e9ec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:25.677037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e5b5c-c567-4a8b-9725-3f955a7e9ec7 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:25.686691 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1333/2662] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:25 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:25.823144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3588422-3d2e-4b8a-a552-181efea0a98a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:25.884103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3588422-3d2e-4b8a-a552-181efea0a98a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:25.888757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:25.889384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.889690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.889995 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] PATH_INFO: `/v3/users/7ecb1b338f17425e917b4977e0f1b27c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:25.891148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ecb1b338f17425e917b4977e0f1b27c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:25.893070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:25.897662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:25.944704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ecb1b338f17425e917b4977e0f1b27c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:25.945348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ecb1b338f17425e917b4977e0f1b27c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:25.953183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ecb1b338f17425e917b4977e0f1b27c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:25.953799 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ecb1b338f17425e917b4977e0f1b27c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:25.960612 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ecb1b338f17425e917b4977e0f1b27c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:25.977628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1330/2663] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:25 2022] DELETE /identity/v3/users/7ecb1b338f17425e917b4977e0f1b27c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:25.982210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d522177-f807-4057-99f6-ecb2b71e19a4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:25.982503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d522177-f807-4057-99f6-ecb2b71e19a4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:25.982781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d522177-f807-4057-99f6-ecb2b71e19a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:26.001783 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9d522177-f807-4057-99f6-ecb2b71e19a4 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:26.003013 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9d522177-f807-4057-99f6-ecb2b71e19a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:26.060011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1334/2664] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:26.069441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:26.071430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:26.126972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060de288-de0b-440d-9ddd-a3fa2adead99 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:26.130629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c3e35-f160-4f5c-953a-a95fedff06f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:26.131224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c3e35-f160-4f5c-953a-a95fedff06f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:26.131501 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c3e35-f160-4f5c-953a-a95fedff06f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:26.131798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c3e35-f160-4f5c-953a-a95fedff06f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:26.141176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c3e35-f160-4f5c-953a-a95fedff06f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:26.142615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c3e35-f160-4f5c-953a-a95fedff06f7 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:26.151696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1331/2665] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:26.420999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d522177-f807-4057-99f6-ecb2b71e19a4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:26.422558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9d522177-f807-4057-99f6-ecb2b71e19a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:26.435366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d522177-f807-4057-99f6-ecb2b71e19a4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:26.440241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:26.440884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:26.441181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:26.441457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] PATH_INFO: `/v3/projects/39cfd41d37c948c183b7586b27bd4189` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:26.442618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39cfd41d37c948c183b7586b27bd4189)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:26.444523 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:26.475402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:26.492344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39cfd41d37c948c183b7586b27bd4189', 'request_id': 'req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.500299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39cfd41d37c948c183b7586b27bd4189', 'request_id': 'req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.506123 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39cfd41d37c948c183b7586b27bd4189', 'request_id': 'req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.517051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] The token cache is being invalidate because project 39cfd41d37c948c183b7586b27bd4189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:26.517337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.517982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.519420 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1335/2666] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:26 2022] DELETE /identity/v3/projects/39cfd41d37c948c183b7586b27bd4189 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:26.523627 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c3e35-f160-4f5c-953a-a95fedff06f7 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:26.533277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c3e35-f160-4f5c-953a-a95fedff06f7 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:26.588568 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:26.589120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:26.589401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:26.589675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] PATH_INFO: `/v3/users/059465771c3f49638074033455b12e1f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:26.590825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] RBAC: Authorizing `identity:delete_user(user_id=059465771c3f49638074033455b12e1f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:26.592751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:26.621482 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:26.639667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059465771c3f49638074033455b12e1f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.640360 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059465771c3f49638074033455b12e1f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.647226 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059465771c3f49638074033455b12e1f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.647830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059465771c3f49638074033455b12e1f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.655190 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0216fecb-e855-42c5-b612-54546591618f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059465771c3f49638074033455b12e1f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:26.672379 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1332/2667] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:26 2022] DELETE /identity/v3/users/059465771c3f49638074033455b12e1f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:27.031137 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:27.037258 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:27.050645 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6554d7-f3c1-4a94-bdd8-b17a3a622da4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:27.108130 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:27.108769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.109051 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.109329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] PATH_INFO: `/v3/projects/4ed7d8a2107d45e6836d8ad52a753a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.110505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed7d8a2107d45e6836d8ad52a753a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:27.112639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:27.164198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed7d8a2107d45e6836d8ad52a753a6f', 'request_id': 'req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:27.174090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed7d8a2107d45e6836d8ad52a753a6f', 'request_id': 'req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:27.179841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed7d8a2107d45e6836d8ad52a753a6f', 'request_id': 'req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:27.191543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] The token cache is being invalidate because project 4ed7d8a2107d45e6836d8ad52a753a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:27.191839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:27.192505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-209b2b57-fdce-41c2-b9fb-6aac65fe75d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:27.194025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1336/2668] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:27 2022] DELETE /identity/v3/projects/4ed7d8a2107d45e6836d8ad52a753a6f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:27.199282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b8679-9615-4c4a-9a0f-6f8d8cd1c918 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.199743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b8679-9615-4c4a-9a0f-6f8d8cd1c918 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.199863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b8679-9615-4c4a-9a0f-6f8d8cd1c918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.221706 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c76b8679-9615-4c4a-9a0f-6f8d8cd1c918 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:27.222981 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c76b8679-9615-4c4a-9a0f-6f8d8cd1c918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:27.293243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1333/2669] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:27.300922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69254866-1120-4f5b-be7b-cdfa7cad062c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.301219 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69254866-1120-4f5b-be7b-cdfa7cad062c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.301497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69254866-1120-4f5b-be7b-cdfa7cad062c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.324033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-69254866-1120-4f5b-be7b-cdfa7cad062c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:27.326483 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-69254866-1120-4f5b-be7b-cdfa7cad062c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:27.337484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1337/2670] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:27.341558 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b8679-9615-4c4a-9a0f-6f8d8cd1c918 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:27.351844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b8679-9615-4c4a-9a0f-6f8d8cd1c918 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:27.356895 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc579554-20df-4f06-8956-8ee17ec4f1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:27.357527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579554-20df-4f06-8956-8ee17ec4f1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.357828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579554-20df-4f06-8956-8ee17ec4f1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.358136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579554-20df-4f06-8956-8ee17ec4f1e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.359105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc579554-20df-4f06-8956-8ee17ec4f1e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:27.359105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc579554-20df-4f06-8956-8ee17ec4f1e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:27.360719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc579554-20df-4f06-8956-8ee17ec4f1e3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:27.377873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1334/2671] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:27.379379 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69254866-1120-4f5b-be7b-cdfa7cad062c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:27.391186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69254866-1120-4f5b-be7b-cdfa7cad062c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:27.396773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7c0bc-b2f9-4c2d-b4a8-3670482aeff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:27.397575 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7c0bc-b2f9-4c2d-b4a8-3670482aeff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.398058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7c0bc-b2f9-4c2d-b4a8-3670482aeff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.398494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7c0bc-b2f9-4c2d-b4a8-3670482aeff9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.399357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7c0bc-b2f9-4c2d-b4a8-3670482aeff9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:27.402567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7c0bc-b2f9-4c2d-b4a8-3670482aeff9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:27.447995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1338/2672] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:27 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:27.449086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc579554-20df-4f06-8956-8ee17ec4f1e3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:27.459832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc579554-20df-4f06-8956-8ee17ec4f1e3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:27.524940 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679406f2-6d5e-494c-b420-26ce632f5a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:27.525545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679406f2-6d5e-494c-b420-26ce632f5a66 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.525823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679406f2-6d5e-494c-b420-26ce632f5a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.526145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679406f2-6d5e-494c-b420-26ce632f5a66 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.526819 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679406f2-6d5e-494c-b420-26ce632f5a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:27.528831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679406f2-6d5e-494c-b420-26ce632f5a66 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:27.553972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1335/2673] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:27 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:27.558778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7c0bc-b2f9-4c2d-b4a8-3670482aeff9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:27.571761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7c0bc-b2f9-4c2d-b4a8-3670482aeff9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:27.578099 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc1f4f-34e9-4e72-ad57-2340f10f9bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:27.578708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc1f4f-34e9-4e72-ad57-2340f10f9bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.579122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc1f4f-34e9-4e72-ad57-2340f10f9bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.579398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc1f4f-34e9-4e72-ad57-2340f10f9bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.580037 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc1f4f-34e9-4e72-ad57-2340f10f9bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:27.581822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc1f4f-34e9-4e72-ad57-2340f10f9bc1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:27.594146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1339/2674] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:35:27.597153 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679406f2-6d5e-494c-b420-26ce632f5a66 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:27.607316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679406f2-6d5e-494c-b420-26ce632f5a66 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:27.612216 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfa7fa-ce06-41bc-bed4-340bb6616741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:27.612762 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfa7fa-ce06-41bc-bed4-340bb6616741 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.613041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfa7fa-ce06-41bc-bed4-340bb6616741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.613311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfa7fa-ce06-41bc-bed4-340bb6616741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.613976 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abfa7fa-ce06-41bc-bed4-340bb6616741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:27.615557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abfa7fa-ce06-41bc-bed4-340bb6616741 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:27.631746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1336/2675] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:27.636566 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc1f4f-34e9-4e72-ad57-2340f10f9bc1 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:27.647530 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc1f4f-34e9-4e72-ad57-2340f10f9bc1 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:27.652389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79439992-926e-4216-be9f-f0f303691cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:27.653215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79439992-926e-4216-be9f-f0f303691cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.653429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79439992-926e-4216-be9f-f0f303691cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.653827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79439992-926e-4216-be9f-f0f303691cd8 admin admin] PATH_INFO: `/v3/projects/52bd875c1399477bbe5a9fa08d8176f7/users/5fa2031d5e8147b78acd2fd17dc69bf8/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.656286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:27.675137 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79439992-926e-4216-be9f-f0f303691cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52bd875c1399477bbe5a9fa08d8176f7, user_id=5fa2031d5e8147b78acd2fd17dc69bf8, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:27.677904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79439992-926e-4216-be9f-f0f303691cd8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:27.693244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1340/2676] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:27 2022] PUT /identity/v3/projects/52bd875c1399477bbe5a9fa08d8176f7/users/5fa2031d5e8147b78acd2fd17dc69bf8/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:27.704619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950abb2-bb6a-447b-a88e-d3ff3038731a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.704890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950abb2-bb6a-447b-a88e-d3ff3038731a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.705214 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950abb2-bb6a-447b-a88e-d3ff3038731a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.734250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:27.747166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3950abb2-bb6a-447b-a88e-d3ff3038731a None None] MFA Rules not processed for user `5fa2031d5e8147b78acd2fd17dc69bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:27.748626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3950abb2-bb6a-447b-a88e-d3ff3038731a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:27.831421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1337/2677] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:27.845142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79439992-926e-4216-be9f-f0f303691cd8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:27.847307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-79439992-926e-4216-be9f-f0f303691cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:27.907032 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79439992-926e-4216-be9f-f0f303691cd8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:27.910604 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f67e8-cad4-4206-881b-97ba8f6328fd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5305f8a5d2b540de91d3caedad454bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:27.911204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f67e8-cad4-4206-881b-97ba8f6328fd service glance] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:27.911478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f67e8-cad4-4206-881b-97ba8f6328fd service glance] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:27.911744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f67e8-cad4-4206-881b-97ba8f6328fd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:27.920358 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f67e8-cad4-4206-881b-97ba8f6328fd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:27.921754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f67e8-cad4-4206-881b-97ba8f6328fd service glance] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:27.931337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1341/2678] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:32.453377 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950abb2-bb6a-447b-a88e-d3ff3038731a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:32.454923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3950abb2-bb6a-447b-a88e-d3ff3038731a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:32.545699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950abb2-bb6a-447b-a88e-d3ff3038731a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:32.550817 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:32.551428 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:32.551707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:32.551979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:32.553714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:32.565954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:32.567517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:32.577075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1338/2679] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:32.693786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f67e8-cad4-4206-881b-97ba8f6328fd service glance] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:32.695198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-873f67e8-cad4-4206-881b-97ba8f6328fd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:32.760980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f67e8-cad4-4206-881b-97ba8f6328fd service glance] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:32.771720 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39531182-4b94-43ca-ae22-a0bd4c9e829d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:32.772644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39531182-4b94-43ca-ae22-a0bd4c9e829d service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:32.773078 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39531182-4b94-43ca-ae22-a0bd4c9e829d service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:32.773526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39531182-4b94-43ca-ae22-a0bd4c9e829d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:32.776827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-39531182-4b94-43ca-ae22-a0bd4c9e829d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:32.840333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39531182-4b94-43ca-ae22-a0bd4c9e829d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:32.841759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39531182-4b94-43ca-ae22-a0bd4c9e829d service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:32.851305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1342/2680] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:36.124620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:36.126545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:36.140822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75305c69-d4d3-4b47-b59e-683b7e523baa service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:36.145490 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:36.146156 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:36.146438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:36.146709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] PATH_INFO: `/v3/users/61a257ce292c4c7fa40fe35dbce1964c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:36.147827 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a257ce292c4c7fa40fe35dbce1964c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:36.150287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:36.196908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a257ce292c4c7fa40fe35dbce1964c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.197571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a257ce292c4c7fa40fe35dbce1964c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.205325 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a257ce292c4c7fa40fe35dbce1964c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.206075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a257ce292c4c7fa40fe35dbce1964c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.213753 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-41de40ed-5ce5-4ac4-b286-435123da7d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a257ce292c4c7fa40fe35dbce1964c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.215852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:36.235327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1339/2681] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:36 2022] DELETE /identity/v3/users/61a257ce292c4c7fa40fe35dbce1964c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:36.417550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39531182-4b94-43ca-ae22-a0bd4c9e829d service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:36.435115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39531182-4b94-43ca-ae22-a0bd4c9e829d service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:36.470226 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:36.499602 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:36.500134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:36.500407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:36.500673 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] PATH_INFO: `/v3/projects/4c3f0c7489be4d2291ec80d969e3a584` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:36.501889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c3f0c7489be4d2291ec80d969e3a584)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:36.503867 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:36.551948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c3f0c7489be4d2291ec80d969e3a584', 'request_id': 'req-07952d76-519c-4226-9f18-965f4f466554'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.559824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c3f0c7489be4d2291ec80d969e3a584', 'request_id': 'req-07952d76-519c-4226-9f18-965f4f466554'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.565877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c3f0c7489be4d2291ec80d969e3a584', 'request_id': 'req-07952d76-519c-4226-9f18-965f4f466554'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.579296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] The token cache is being invalidate because project 4c3f0c7489be4d2291ec80d969e3a584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:36.579792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.581126 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07952d76-519c-4226-9f18-965f4f466554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:36.583012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1343/2682] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:36 2022] DELETE /identity/v3/projects/4c3f0c7489be4d2291ec80d969e3a584 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:36.588030 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a455ca-0208-405f-815a-af218cd105dc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:36.588318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a455ca-0208-405f-815a-af218cd105dc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:36.588593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a455ca-0208-405f-815a-af218cd105dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:36.609801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e4a455ca-0208-405f-815a-af218cd105dc None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:36.611423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e4a455ca-0208-405f-815a-af218cd105dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:36.795913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1340/2683] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:36.803649 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ab702-50e9-4be4-9fe3-4ad0ae5f1a48 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:36.803939 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ab702-50e9-4be4-9fe3-4ad0ae5f1a48 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:36.804220 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ab702-50e9-4be4-9fe3-4ad0ae5f1a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:36.824921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f06ab702-50e9-4be4-9fe3-4ad0ae5f1a48 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:36.826204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f06ab702-50e9-4be4-9fe3-4ad0ae5f1a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:36.834012 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1344/2684] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:36.838736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a455ca-0208-405f-815a-af218cd105dc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:36.848557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a455ca-0208-405f-815a-af218cd105dc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:36.853599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df015fe-1901-4a3e-81a5-9668604c0385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:36.854220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df015fe-1901-4a3e-81a5-9668604c0385 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:36.854494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df015fe-1901-4a3e-81a5-9668604c0385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:36.854757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df015fe-1901-4a3e-81a5-9668604c0385 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:36.855392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df015fe-1901-4a3e-81a5-9668604c0385 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:36.855685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df015fe-1901-4a3e-81a5-9668604c0385 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:36.857253 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df015fe-1901-4a3e-81a5-9668604c0385 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:36.869101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1341/2685] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:36.873508 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ab702-50e9-4be4-9fe3-4ad0ae5f1a48 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:36.882692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ab702-50e9-4be4-9fe3-4ad0ae5f1a48 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:36.887319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ac486-58f9-472a-951e-d13a99377c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:36.887920 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ac486-58f9-472a-951e-d13a99377c94 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:36.888203 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ac486-58f9-472a-951e-d13a99377c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:36.888495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ac486-58f9-472a-951e-d13a99377c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:36.889129 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ac486-58f9-472a-951e-d13a99377c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:36.891063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ac486-58f9-472a-951e-d13a99377c94 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:36.919314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1345/2686] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:36 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:36.923619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df015fe-1901-4a3e-81a5-9668604c0385 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:36.933812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df015fe-1901-4a3e-81a5-9668604c0385 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:37.000659 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4de66-b0b6-402c-8056-588583226b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:37.001276 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4de66-b0b6-402c-8056-588583226b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:37.001552 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4de66-b0b6-402c-8056-588583226b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:37.001822 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4de66-b0b6-402c-8056-588583226b31 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:37.002500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4de66-b0b6-402c-8056-588583226b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:37.004423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4de66-b0b6-402c-8056-588583226b31 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:37.027137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1342/2687] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:36 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:37.031270 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ac486-58f9-472a-951e-d13a99377c94 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:37.042112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ac486-58f9-472a-951e-d13a99377c94 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:37.046729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6ea8b-0eab-423c-87e8-af0c69c73571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:37.047272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ea8b-0eab-423c-87e8-af0c69c73571 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:37.047531 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ea8b-0eab-423c-87e8-af0c69c73571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:37.047800 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ea8b-0eab-423c-87e8-af0c69c73571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:37.048461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6ea8b-0eab-423c-87e8-af0c69c73571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:37.050047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6ea8b-0eab-423c-87e8-af0c69c73571 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:37.059699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1346/2688] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:37.063972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4de66-b0b6-402c-8056-588583226b31 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:37.075022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4de66-b0b6-402c-8056-588583226b31 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:37.079731 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daa885-155e-4e84-8453-8d7c1443ae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:37.080332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daa885-155e-4e84-8453-8d7c1443ae80 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:37.080612 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daa885-155e-4e84-8453-8d7c1443ae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:37.080885 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daa885-155e-4e84-8453-8d7c1443ae80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:37.081522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daa885-155e-4e84-8453-8d7c1443ae80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:37.083136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daa885-155e-4e84-8453-8d7c1443ae80 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:37.094498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1343/2689] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:37.098714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6ea8b-0eab-423c-87e8-af0c69c73571 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:37.108739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6ea8b-0eab-423c-87e8-af0c69c73571 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:37.113526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ac672-72fb-4838-9b28-d10c632826f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:37.114193 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ac672-72fb-4838-9b28-d10c632826f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:37.114471 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ac672-72fb-4838-9b28-d10c632826f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:37.114744 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ac672-72fb-4838-9b28-d10c632826f7 admin admin] PATH_INFO: `/v3/projects/2efdef788aed493aba59c9d25dbe25c5/users/6d6f67f5bf5c49fd8a8636542aa14f61/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:37.128880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ac672-72fb-4838-9b28-d10c632826f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2efdef788aed493aba59c9d25dbe25c5, user_id=6d6f67f5bf5c49fd8a8636542aa14f61, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:37.131571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ac672-72fb-4838-9b28-d10c632826f7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:37.146902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1347/2690] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:37 2022] PUT /identity/v3/projects/2efdef788aed493aba59c9d25dbe25c5/users/6d6f67f5bf5c49fd8a8636542aa14f61/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:37.159082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808108be-3766-475f-b901-b5add6124209 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:37.159573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808108be-3766-475f-b901-b5add6124209 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:37.159673 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808108be-3766-475f-b901-b5add6124209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:37.187085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-808108be-3766-475f-b901-b5add6124209 None None] MFA Rules not processed for user `6d6f67f5bf5c49fd8a8636542aa14f61`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:37.188294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-808108be-3766-475f-b901-b5add6124209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:37.226232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:37.254260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1344/2691] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:37.266318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ac672-72fb-4838-9b28-d10c632826f7 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:37.267963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b21ac672-72fb-4838-9b28-d10c632826f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:37.333291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:37.374279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ac672-72fb-4838-9b28-d10c632826f7 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:37.379742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee011a6-ea6e-46c3-aa5c-33f4bf0f4913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:37.380336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee011a6-ea6e-46c3-aa5c-33f4bf0f4913 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:37.380612 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee011a6-ea6e-46c3-aa5c-33f4bf0f4913 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:37.380882 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee011a6-ea6e-46c3-aa5c-33f4bf0f4913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:37.388475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee011a6-ea6e-46c3-aa5c-33f4bf0f4913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:37.390059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee011a6-ea6e-46c3-aa5c-33f4bf0f4913 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:37.398550 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1348/2692] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:37.634808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808108be-3766-475f-b901-b5add6124209 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:37.703577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808108be-3766-475f-b901-b5add6124209 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:37.708042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:37.708672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:37.708950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:37.709222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] PATH_INFO: `/v3/users/6d6f67f5bf5c49fd8a8636542aa14f61` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:37.710410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d6f67f5bf5c49fd8a8636542aa14f61)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:37.712329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:37.765236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d6f67f5bf5c49fd8a8636542aa14f61'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:37.765908 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d6f67f5bf5c49fd8a8636542aa14f61'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:37.773462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d6f67f5bf5c49fd8a8636542aa14f61'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:37.774154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d6f67f5bf5c49fd8a8636542aa14f61'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:37.781095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d6f67f5bf5c49fd8a8636542aa14f61'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:37.797855 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1345/2693] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:37 2022] DELETE /identity/v3/users/6d6f67f5bf5c49fd8a8636542aa14f61 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:37.802631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e774e-719a-4387-a43a-c5ab8735a976 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:37.803146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e774e-719a-4387-a43a-c5ab8735a976 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:37.803377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e774e-719a-4387-a43a-c5ab8735a976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:37.823249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9e2e774e-719a-4387-a43a-c5ab8735a976 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:37.824509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9e2e774e-719a-4387-a43a-c5ab8735a976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:37.887628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1349/2694] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:37.897215 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:37.898669 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:37.965878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:37.983395 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5021f4-2e58-45e3-9bf5-bed7fc791b16 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:37.987314 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee39da2-fb8e-4922-a92d-cb14266e496b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:37.987914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39da2-fb8e-4922-a92d-cb14266e496b service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:37.988191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39da2-fb8e-4922-a92d-cb14266e496b service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:37.988460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39da2-fb8e-4922-a92d-cb14266e496b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:37.999526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee39da2-fb8e-4922-a92d-cb14266e496b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:38.002124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee39da2-fb8e-4922-a92d-cb14266e496b service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:38.015142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1346/2695] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:35:38.291585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e774e-719a-4387-a43a-c5ab8735a976 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:38.293166 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9e2e774e-719a-4387-a43a-c5ab8735a976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:38.307767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e774e-719a-4387-a43a-c5ab8735a976 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:38.314108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:38.315329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.316045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.316553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] PATH_INFO: `/v3/projects/2efdef788aed493aba59c9d25dbe25c5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.318058 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2efdef788aed493aba59c9d25dbe25c5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:38.321052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:38.371338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2efdef788aed493aba59c9d25dbe25c5', 'request_id': 'req-07580ddd-47bd-4484-93a0-43719ba14968'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:38.378521 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2efdef788aed493aba59c9d25dbe25c5', 'request_id': 'req-07580ddd-47bd-4484-93a0-43719ba14968'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:38.384205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2efdef788aed493aba59c9d25dbe25c5', 'request_id': 'req-07580ddd-47bd-4484-93a0-43719ba14968'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:38.395340 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] The token cache is being invalidate because project 2efdef788aed493aba59c9d25dbe25c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:38.395629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:38.396287 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-07580ddd-47bd-4484-93a0-43719ba14968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:38.397752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1350/2696] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:38 2022] DELETE /identity/v3/projects/2efdef788aed493aba59c9d25dbe25c5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:38.403165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc2789-774c-4dc2-82bf-745841d5f7d5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.403999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc2789-774c-4dc2-82bf-745841d5f7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.403999 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc2789-774c-4dc2-82bf-745841d5f7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.424036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f6cc2789-774c-4dc2-82bf-745841d5f7d5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:38.425151 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f6cc2789-774c-4dc2-82bf-745841d5f7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:38.483974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1347/2697] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:38.491921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873a3b1-c1b4-4009-8a4f-11e6a1a47830 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.492211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873a3b1-c1b4-4009-8a4f-11e6a1a47830 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.492487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873a3b1-c1b4-4009-8a4f-11e6a1a47830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.513539 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2873a3b1-c1b4-4009-8a4f-11e6a1a47830 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:38.514788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2873a3b1-c1b4-4009-8a4f-11e6a1a47830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:38.522775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1351/2698] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:38.527007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc2789-774c-4dc2-82bf-745841d5f7d5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:38.536415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc2789-774c-4dc2-82bf-745841d5f7d5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:38.540907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787742d-8dd2-49f0-b9e0-0d60aae9f367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:38.541740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787742d-8dd2-49f0-b9e0-0d60aae9f367 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.542152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787742d-8dd2-49f0-b9e0-0d60aae9f367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.542545 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787742d-8dd2-49f0-b9e0-0d60aae9f367 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.543270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787742d-8dd2-49f0-b9e0-0d60aae9f367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:38.543735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787742d-8dd2-49f0-b9e0-0d60aae9f367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:38.546059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787742d-8dd2-49f0-b9e0-0d60aae9f367 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:38.558491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1348/2699] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:38.562933 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873a3b1-c1b4-4009-8a4f-11e6a1a47830 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:38.572626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873a3b1-c1b4-4009-8a4f-11e6a1a47830 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:38.577244 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa843-ab62-4a80-9cbf-3141902fe39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:38.577835 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa843-ab62-4a80-9cbf-3141902fe39d admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.578146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa843-ab62-4a80-9cbf-3141902fe39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.578417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa843-ab62-4a80-9cbf-3141902fe39d admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.579045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faa843-ab62-4a80-9cbf-3141902fe39d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:38.580911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faa843-ab62-4a80-9cbf-3141902fe39d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:38.609271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1352/2700] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:38 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:38.613786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787742d-8dd2-49f0-b9e0-0d60aae9f367 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:38.627247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787742d-8dd2-49f0-b9e0-0d60aae9f367 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:38.694075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88424c9-8fc9-438c-b929-928af0adcd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:38.694679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88424c9-8fc9-438c-b929-928af0adcd22 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.694955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88424c9-8fc9-438c-b929-928af0adcd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.695224 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88424c9-8fc9-438c-b929-928af0adcd22 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.695848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88424c9-8fc9-438c-b929-928af0adcd22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:38.698372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88424c9-8fc9-438c-b929-928af0adcd22 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:38.723148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1349/2701] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:38 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:38.727241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa843-ab62-4a80-9cbf-3141902fe39d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:38.738257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa843-ab62-4a80-9cbf-3141902fe39d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:38.743038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2102e4-839e-4829-bde0-b6c51e14d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:38.744227 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2102e4-839e-4829-bde0-b6c51e14d822 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.744506 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2102e4-839e-4829-bde0-b6c51e14d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.744778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2102e4-839e-4829-bde0-b6c51e14d822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.745410 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2102e4-839e-4829-bde0-b6c51e14d822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:38.747045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2102e4-839e-4829-bde0-b6c51e14d822 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:38.759699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1353/2702] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:38.760835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88424c9-8fc9-438c-b929-928af0adcd22 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:38.771016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88424c9-8fc9-438c-b929-928af0adcd22 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:38.775678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9219fc4-cdb2-4333-a6c5-f6da8e367dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:38.776273 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9219fc4-cdb2-4333-a6c5-f6da8e367dea admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.776554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9219fc4-cdb2-4333-a6c5-f6da8e367dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.776843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9219fc4-cdb2-4333-a6c5-f6da8e367dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.777470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9219fc4-cdb2-4333-a6c5-f6da8e367dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:38.779097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9219fc4-cdb2-4333-a6c5-f6da8e367dea admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:38.789547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1350/2703] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:35:38.794271 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2102e4-839e-4829-bde0-b6c51e14d822 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:38.800659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:38.804454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2102e4-839e-4829-bde0-b6c51e14d822 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:38.810674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5705d-1a5e-43ff-bde4-9230fb575e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:38.811491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5705d-1a5e-43ff-bde4-9230fb575e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.811939 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5705d-1a5e-43ff-bde4-9230fb575e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.812377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5705d-1a5e-43ff-bde4-9230fb575e61 admin admin] PATH_INFO: `/v3/projects/89560c5070e6467b809a399da52b908b/users/60a385604aab48e0a1d640f06a5a70be/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.827312 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5705d-1a5e-43ff-bde4-9230fb575e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89560c5070e6467b809a399da52b908b, user_id=60a385604aab48e0a1d640f06a5a70be, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:38.830041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5705d-1a5e-43ff-bde4-9230fb575e61 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:38.843336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1354/2704] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:38 2022] PUT /identity/v3/projects/89560c5070e6467b809a399da52b908b/users/60a385604aab48e0a1d640f06a5a70be/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:38.853832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35e4df-2777-4f56-bf76-3ea981dd0b7e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:38.854142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35e4df-2777-4f56-bf76-3ea981dd0b7e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:38.854438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35e4df-2777-4f56-bf76-3ea981dd0b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:38.881449 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fe35e4df-2777-4f56-bf76-3ea981dd0b7e None None] MFA Rules not processed for user `60a385604aab48e0a1d640f06a5a70be`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:38.882679 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fe35e4df-2777-4f56-bf76-3ea981dd0b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:38.915459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:38.947274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1351/2705] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:38.951761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5705d-1a5e-43ff-bde4-9230fb575e61 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:39.017584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5705d-1a5e-43ff-bde4-9230fb575e61 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:39.022318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2242bf-320f-4660-862e-92029e6ee9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:39.022917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2242bf-320f-4660-862e-92029e6ee9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:39.023194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2242bf-320f-4660-862e-92029e6ee9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:39.023465 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2242bf-320f-4660-862e-92029e6ee9d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:39.024082 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2242bf-320f-4660-862e-92029e6ee9d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:39.026719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2242bf-320f-4660-862e-92029e6ee9d0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:39.052988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1355/2706] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:38 2022] POST /identity/v3/projects => generated 361 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:39.057272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35e4df-2777-4f56-bf76-3ea981dd0b7e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:39.067741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35e4df-2777-4f56-bf76-3ea981dd0b7e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:39.131228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c830928-6737-434b-bc6a-c401c5e8fe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:39.131842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c830928-6737-434b-bc6a-c401c5e8fe64 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:39.132118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c830928-6737-434b-bc6a-c401c5e8fe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:39.132387 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c830928-6737-434b-bc6a-c401c5e8fe64 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:39.133019 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c830928-6737-434b-bc6a-c401c5e8fe64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:39.135851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c830928-6737-434b-bc6a-c401c5e8fe64 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:39.162237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1352/2707] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:39 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:39.163457 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2242bf-320f-4660-862e-92029e6ee9d0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:39.174690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2242bf-320f-4660-862e-92029e6ee9d0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:39.179570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd61d5-ec5d-4122-8ab3-eecf276c7a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:39.180162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd61d5-ec5d-4122-8ab3-eecf276c7a4a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:39.180442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd61d5-ec5d-4122-8ab3-eecf276c7a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:39.180716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd61d5-ec5d-4122-8ab3-eecf276c7a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:39.181886 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd61d5-ec5d-4122-8ab3-eecf276c7a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:39.183562 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd61d5-ec5d-4122-8ab3-eecf276c7a4a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:39.193924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1356/2708] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:39.198664 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c830928-6737-434b-bc6a-c401c5e8fe64 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:39.208236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c830928-6737-434b-bc6a-c401c5e8fe64 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:39.212723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654c71-d557-4f6a-8740-bc244069acf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:39.213356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654c71-d557-4f6a-8740-bc244069acf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:39.213636 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654c71-d557-4f6a-8740-bc244069acf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:39.213904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654c71-d557-4f6a-8740-bc244069acf9 admin admin] PATH_INFO: `/v3/projects/5312de425d89440a8f2727f887755c09/users/e3e66f1294a4417da95a8debf2eb8944/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:39.228308 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a654c71-d557-4f6a-8740-bc244069acf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5312de425d89440a8f2727f887755c09, user_id=e3e66f1294a4417da95a8debf2eb8944, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:39.230972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a654c71-d557-4f6a-8740-bc244069acf9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:39.244430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1353/2709] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:39 2022] PUT /identity/v3/projects/5312de425d89440a8f2727f887755c09/users/e3e66f1294a4417da95a8debf2eb8944/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:39.248559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd61d5-ec5d-4122-8ab3-eecf276c7a4a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:39.258387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd61d5-ec5d-4122-8ab3-eecf276c7a4a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:39.312638 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0c1a2-dc80-4b14-89ce-3ac2ff7a09e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:39.313225 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0c1a2-dc80-4b14-89ce-3ac2ff7a09e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:39.313502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0c1a2-dc80-4b14-89ce-3ac2ff7a09e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:39.313774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0c1a2-dc80-4b14-89ce-3ac2ff7a09e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:39.314427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0c1a2-dc80-4b14-89ce-3ac2ff7a09e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:39.315982 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0c1a2-dc80-4b14-89ce-3ac2ff7a09e2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:39.326019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1357/2710] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:39.330350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654c71-d557-4f6a-8740-bc244069acf9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:39.340613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654c71-d557-4f6a-8740-bc244069acf9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:39.345166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c74ada-ff2a-438b-a5f1-90cdc2318cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:39.345806 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c74ada-ff2a-438b-a5f1-90cdc2318cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:39.346122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c74ada-ff2a-438b-a5f1-90cdc2318cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:39.346392 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c74ada-ff2a-438b-a5f1-90cdc2318cb7 admin admin] PATH_INFO: `/v3/projects/5312de425d89440a8f2727f887755c09/users/e3e66f1294a4417da95a8debf2eb8944/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:39.348282 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c74ada-ff2a-438b-a5f1-90cdc2318cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5312de425d89440a8f2727f887755c09, user_id=e3e66f1294a4417da95a8debf2eb8944, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:39.351114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c74ada-ff2a-438b-a5f1-90cdc2318cb7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:39.362590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1354/2711] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:39 2022] PUT /identity/v3/projects/5312de425d89440a8f2727f887755c09/users/e3e66f1294a4417da95a8debf2eb8944/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:39.375296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23dabe-809f-43d9-941c-311e30019c44 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:39.375584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23dabe-809f-43d9-941c-311e30019c44 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:39.375858 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23dabe-809f-43d9-941c-311e30019c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:39.403953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8c23dabe-809f-43d9-941c-311e30019c44 None None] MFA Rules not processed for user `e3e66f1294a4417da95a8debf2eb8944`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:39.405162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8c23dabe-809f-43d9-941c-311e30019c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:39.444529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:39.473728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1358/2712] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:39.485097 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c74ada-ff2a-438b-a5f1-90cdc2318cb7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:39.487177 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-51c74ada-ff2a-438b-a5f1-90cdc2318cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:39.543910 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c74ada-ff2a-438b-a5f1-90cdc2318cb7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:39.547584 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302d766-ab3a-4bf0-858d-a7cdbdef4bea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:39.548183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302d766-ab3a-4bf0-858d-a7cdbdef4bea service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:39.548461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302d766-ab3a-4bf0-858d-a7cdbdef4bea service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:39.548737 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302d766-ab3a-4bf0-858d-a7cdbdef4bea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:39.558221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302d766-ab3a-4bf0-858d-a7cdbdef4bea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:39.559676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302d766-ab3a-4bf0-858d-a7cdbdef4bea service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:39.568804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1355/2713] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:39.647221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23dabe-809f-43d9-941c-311e30019c44 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:39.657181 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23dabe-809f-43d9-941c-311e30019c44 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:39.660847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3501b25-0061-4db1-a557-508fc10c7559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:39.661440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3501b25-0061-4db1-a557-508fc10c7559 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:39.661718 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3501b25-0061-4db1-a557-508fc10c7559 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:39.662024 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3501b25-0061-4db1-a557-508fc10c7559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:39.718131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3501b25-0061-4db1-a557-508fc10c7559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:39.719534 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3501b25-0061-4db1-a557-508fc10c7559 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:39.728335 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1359/2714] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:42.454461 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302d766-ab3a-4bf0-858d-a7cdbdef4bea service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:42.488063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:42.517674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302d766-ab3a-4bf0-858d-a7cdbdef4bea service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:42.522148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:42.522763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:42.523034 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:42.523300 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] PATH_INFO: `/v3/users/60a385604aab48e0a1d640f06a5a70be` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:42.524384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a385604aab48e0a1d640f06a5a70be)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:42.526344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:42.570752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a385604aab48e0a1d640f06a5a70be'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:42.571370 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a385604aab48e0a1d640f06a5a70be'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:42.578445 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a385604aab48e0a1d640f06a5a70be'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:42.579054 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a385604aab48e0a1d640f06a5a70be'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:42.586070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a385604aab48e0a1d640f06a5a70be'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:42.605792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1356/2715] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:42 2022] DELETE /identity/v3/users/60a385604aab48e0a1d640f06a5a70be => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:42.610579 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa2dd8-7b26-4965-a8c2-88b44c1433c9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:42.611314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa2dd8-7b26-4965-a8c2-88b44c1433c9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:42.611426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa2dd8-7b26-4965-a8c2-88b44c1433c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:42.630473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d8aa2dd8-7b26-4965-a8c2-88b44c1433c9 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:42.631674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d8aa2dd8-7b26-4965-a8c2-88b44c1433c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:42.688958 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1360/2716] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:42.698761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:42.700277 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:42.756342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80f187-1c28-4be2-af96-524b50b0c0a2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:42.759891 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab487af-1870-479e-bad9-f4196151370a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:42.760494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab487af-1870-479e-bad9-f4196151370a service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:42.760770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab487af-1870-479e-bad9-f4196151370a service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:42.761039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab487af-1870-479e-bad9-f4196151370a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:42.769833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab487af-1870-479e-bad9-f4196151370a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:42.771296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab487af-1870-479e-bad9-f4196151370a service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:42.781056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1357/2717] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:43.037738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa2dd8-7b26-4965-a8c2-88b44c1433c9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:43.039186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d8aa2dd8-7b26-4965-a8c2-88b44c1433c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:43.052526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa2dd8-7b26-4965-a8c2-88b44c1433c9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:43.057133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:43.057793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:43.058184 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:43.058463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] PATH_INFO: `/v3/projects/89560c5070e6467b809a399da52b908b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:43.059624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=89560c5070e6467b809a399da52b908b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:43.061662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:43.110183 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89560c5070e6467b809a399da52b908b', 'request_id': 'req-4ee90dbe-41b5-4f25-91db-6901b30a9dca'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.117479 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89560c5070e6467b809a399da52b908b', 'request_id': 'req-4ee90dbe-41b5-4f25-91db-6901b30a9dca'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.123054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89560c5070e6467b809a399da52b908b', 'request_id': 'req-4ee90dbe-41b5-4f25-91db-6901b30a9dca'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.130047 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:43.135268 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] The token cache is being invalidate because project 89560c5070e6467b809a399da52b908b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:43.136086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.136823 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.138827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1361/2718] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:43 2022] DELETE /identity/v3/projects/89560c5070e6467b809a399da52b908b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:43.142828 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab487af-1870-479e-bad9-f4196151370a service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:43.152978 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab487af-1870-479e-bad9-f4196151370a service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:43.200520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:43.210598 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:43.211236 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:43.211727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:43.212011 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] PATH_INFO: `/v3/users/e3e66f1294a4417da95a8debf2eb8944` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:43.213137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e66f1294a4417da95a8debf2eb8944)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:43.216100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:43.265031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e66f1294a4417da95a8debf2eb8944'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.265680 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e66f1294a4417da95a8debf2eb8944'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.272427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e66f1294a4417da95a8debf2eb8944'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.273137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e66f1294a4417da95a8debf2eb8944'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.280830 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f337e0fb-52df-4135-8137-b07bdca7792a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e66f1294a4417da95a8debf2eb8944'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.298810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1358/2719] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:43 2022] DELETE /identity/v3/users/e3e66f1294a4417da95a8debf2eb8944 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:43.575732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:43.577321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:43.594087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee90dbe-41b5-4f25-91db-6901b30a9dca admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:43.651223 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:43.651880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:43.652124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:43.652398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] PATH_INFO: `/v3/projects/5312de425d89440a8f2727f887755c09` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:43.653515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5312de425d89440a8f2727f887755c09)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:43.655438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:43.700743 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5312de425d89440a8f2727f887755c09', 'request_id': 'req-3db0cf28-ec71-44dd-bf67-0912a193690f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.707742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5312de425d89440a8f2727f887755c09', 'request_id': 'req-3db0cf28-ec71-44dd-bf67-0912a193690f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.713186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5312de425d89440a8f2727f887755c09', 'request_id': 'req-3db0cf28-ec71-44dd-bf67-0912a193690f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.724299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] The token cache is being invalidate because project 5312de425d89440a8f2727f887755c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:43.724593 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.725221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3db0cf28-ec71-44dd-bf67-0912a193690f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:43.726742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1362/2720] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:43 2022] DELETE /identity/v3/projects/5312de425d89440a8f2727f887755c09 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:43.731657 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03b550-38b2-4c8b-9ab7-7f187c358a91 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:43.732109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03b550-38b2-4c8b-9ab7-7f187c358a91 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:43.732228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03b550-38b2-4c8b-9ab7-7f187c358a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:43.752423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cc03b550-38b2-4c8b-9ab7-7f187c358a91 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:43.753661 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cc03b550-38b2-4c8b-9ab7-7f187c358a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:43.811729 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1359/2721] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:43.819425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01528a8-e37c-47ea-af49-330334140f08 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:43.819712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01528a8-e37c-47ea-af49-330334140f08 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:43.819986 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01528a8-e37c-47ea-af49-330334140f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:43.839402 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c01528a8-e37c-47ea-af49-330334140f08 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:43.840590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c01528a8-e37c-47ea-af49-330334140f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:43.848338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1363/2722] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:43.852618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03b550-38b2-4c8b-9ab7-7f187c358a91 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:43.862916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03b550-38b2-4c8b-9ab7-7f187c358a91 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:43.867402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ebc4f-0555-4b37-84a7-dbf37d4cf99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:43.867989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ebc4f-0555-4b37-84a7-dbf37d4cf99c admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:43.868268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ebc4f-0555-4b37-84a7-dbf37d4cf99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:43.868534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ebc4f-0555-4b37-84a7-dbf37d4cf99c admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:43.869168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ebc4f-0555-4b37-84a7-dbf37d4cf99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:43.869468 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ebc4f-0555-4b37-84a7-dbf37d4cf99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:43.871681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ebc4f-0555-4b37-84a7-dbf37d4cf99c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:43.883210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1360/2723] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:43.887588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01528a8-e37c-47ea-af49-330334140f08 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:43.896598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01528a8-e37c-47ea-af49-330334140f08 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:43.901120 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056f5bf-dc42-41b5-bf05-e642e4c340ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:43.901707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056f5bf-dc42-41b5-bf05-e642e4c340ef admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:43.902016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056f5bf-dc42-41b5-bf05-e642e4c340ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:43.902291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056f5bf-dc42-41b5-bf05-e642e4c340ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:43.902908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056f5bf-dc42-41b5-bf05-e642e4c340ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:43.904769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056f5bf-dc42-41b5-bf05-e642e4c340ef admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:43.931502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1364/2724] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:43 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:43.936194 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ebc4f-0555-4b37-84a7-dbf37d4cf99c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:43.945775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ebc4f-0555-4b37-84a7-dbf37d4cf99c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.000477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff5917-cfc7-4e86-9e23-f62057d64e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.001068 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff5917-cfc7-4e86-9e23-f62057d64e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.001343 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff5917-cfc7-4e86-9e23-f62057d64e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.001613 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff5917-cfc7-4e86-9e23-f62057d64e15 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.002263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff5917-cfc7-4e86-9e23-f62057d64e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.004138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff5917-cfc7-4e86-9e23-f62057d64e15 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.011844 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:44.026147 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1361/2725] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:43 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:44.030248 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056f5bf-dc42-41b5-bf05-e642e4c340ef admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.040428 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056f5bf-dc42-41b5-bf05-e642e4c340ef admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.045600 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e19d9-8b7e-4e97-95b5-c920d7d0aabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.046235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e19d9-8b7e-4e97-95b5-c920d7d0aabb admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.046510 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e19d9-8b7e-4e97-95b5-c920d7d0aabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.046785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e19d9-8b7e-4e97-95b5-c920d7d0aabb admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.047497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e19d9-8b7e-4e97-95b5-c920d7d0aabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.049190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e19d9-8b7e-4e97-95b5-c920d7d0aabb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.059541 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1365/2726] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:44.063572 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff5917-cfc7-4e86-9e23-f62057d64e15 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.073118 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff5917-cfc7-4e86-9e23-f62057d64e15 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.077671 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36142d-57d9-411c-b86f-aaaa8f71ddcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.078289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36142d-57d9-411c-b86f-aaaa8f71ddcb admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.078571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36142d-57d9-411c-b86f-aaaa8f71ddcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.078842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36142d-57d9-411c-b86f-aaaa8f71ddcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.079497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36142d-57d9-411c-b86f-aaaa8f71ddcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.081071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36142d-57d9-411c-b86f-aaaa8f71ddcb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.091372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1362/2727] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:44.095414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e19d9-8b7e-4e97-95b5-c920d7d0aabb admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.105805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e19d9-8b7e-4e97-95b5-c920d7d0aabb admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.110516 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5fa65-e19e-4e9d-9d7e-59ec491d9fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.111448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5fa65-e19e-4e9d-9d7e-59ec491d9fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.111749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5fa65-e19e-4e9d-9d7e-59ec491d9fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.112025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5fa65-e19e-4e9d-9d7e-59ec491d9fb8 admin admin] PATH_INFO: `/v3/projects/b0b0960fc65f440abb014139999dd7ad/users/5df88dc067484b1c85e6e6081294df14/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.126898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5fa65-e19e-4e9d-9d7e-59ec491d9fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b0960fc65f440abb014139999dd7ad, user_id=5df88dc067484b1c85e6e6081294df14, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.129515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5fa65-e19e-4e9d-9d7e-59ec491d9fb8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.142636 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1366/2728] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:44 2022] PUT /identity/v3/projects/b0b0960fc65f440abb014139999dd7ad/users/5df88dc067484b1c85e6e6081294df14/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:44.154717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67be2a-626f-4aad-9b9d-95c2b1ebf0ec None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.154998 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67be2a-626f-4aad-9b9d-95c2b1ebf0ec None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.155265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67be2a-626f-4aad-9b9d-95c2b1ebf0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.182518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-eb67be2a-626f-4aad-9b9d-95c2b1ebf0ec None None] MFA Rules not processed for user `5df88dc067484b1c85e6e6081294df14`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:44.183707 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-eb67be2a-626f-4aad-9b9d-95c2b1ebf0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:44.245074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1363/2729] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:44.249564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5fa65-e19e-4e9d-9d7e-59ec491d9fb8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.275584 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:44.312117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5fa65-e19e-4e9d-9d7e-59ec491d9fb8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.316739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc6976-ab1d-42da-ba04-f1db88e281ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.317279 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc6976-ab1d-42da-ba04-f1db88e281ad admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.317556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc6976-ab1d-42da-ba04-f1db88e281ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.317826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc6976-ab1d-42da-ba04-f1db88e281ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.318465 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc6976-ab1d-42da-ba04-f1db88e281ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.320343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc6976-ab1d-42da-ba04-f1db88e281ad admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.346791 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1367/2730] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:44 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:44.351322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67be2a-626f-4aad-9b9d-95c2b1ebf0ec None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.365045 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67be2a-626f-4aad-9b9d-95c2b1ebf0ec None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.429252 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfbbbd-13be-4f3a-99ab-9920955ac366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.429864 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfbbbd-13be-4f3a-99ab-9920955ac366 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.430189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfbbbd-13be-4f3a-99ab-9920955ac366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.430447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfbbbd-13be-4f3a-99ab-9920955ac366 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.431086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfbbbd-13be-4f3a-99ab-9920955ac366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.433024 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfbbbd-13be-4f3a-99ab-9920955ac366 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.455628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1364/2731] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:44 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:44.459726 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc6976-ab1d-42da-ba04-f1db88e281ad admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.471460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc6976-ab1d-42da-ba04-f1db88e281ad admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.483572 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262b1ca-a695-4aaf-8876-73894ff87fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.484187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262b1ca-a695-4aaf-8876-73894ff87fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.484470 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262b1ca-a695-4aaf-8876-73894ff87fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.485266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262b1ca-a695-4aaf-8876-73894ff87fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.486197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262b1ca-a695-4aaf-8876-73894ff87fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.487821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262b1ca-a695-4aaf-8876-73894ff87fa1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.498926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1368/2732] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:44.503232 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfbbbd-13be-4f3a-99ab-9920955ac366 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.513145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfbbbd-13be-4f3a-99ab-9920955ac366 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.524637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec65313-8f1c-48bf-b86b-e6c58f964579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.524637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec65313-8f1c-48bf-b86b-e6c58f964579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.524637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec65313-8f1c-48bf-b86b-e6c58f964579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.524637 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec65313-8f1c-48bf-b86b-e6c58f964579 admin admin] PATH_INFO: `/v3/projects/9d7a20086cdb4bfaa67e44d21691b95a/users/a6f88dc374e6446a8904689f589c939c/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.535305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec65313-8f1c-48bf-b86b-e6c58f964579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d7a20086cdb4bfaa67e44d21691b95a, user_id=a6f88dc374e6446a8904689f589c939c, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.537964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec65313-8f1c-48bf-b86b-e6c58f964579 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.551472 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1365/2733] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:44 2022] PUT /identity/v3/projects/9d7a20086cdb4bfaa67e44d21691b95a/users/a6f88dc374e6446a8904689f589c939c/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:44.555436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262b1ca-a695-4aaf-8876-73894ff87fa1 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.566598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262b1ca-a695-4aaf-8876-73894ff87fa1 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.627165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cf91d-df30-4e28-988d-b6020801477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.627768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cf91d-df30-4e28-988d-b6020801477a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.628046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cf91d-df30-4e28-988d-b6020801477a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.628344 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cf91d-df30-4e28-988d-b6020801477a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.629087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cf91d-df30-4e28-988d-b6020801477a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.630741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cf91d-df30-4e28-988d-b6020801477a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.646362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1369/2734] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:35:44.646491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec65313-8f1c-48bf-b86b-e6c58f964579 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.656689 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec65313-8f1c-48bf-b86b-e6c58f964579 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.661477 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1714c2a-e7d8-4bba-bc3d-383d4a0aa87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.662081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1714c2a-e7d8-4bba-bc3d-383d4a0aa87c admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.662363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1714c2a-e7d8-4bba-bc3d-383d4a0aa87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.662640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1714c2a-e7d8-4bba-bc3d-383d4a0aa87c admin admin] PATH_INFO: `/v3/projects/9d7a20086cdb4bfaa67e44d21691b95a/users/a6f88dc374e6446a8904689f589c939c/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.664573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1714c2a-e7d8-4bba-bc3d-383d4a0aa87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d7a20086cdb4bfaa67e44d21691b95a, user_id=a6f88dc374e6446a8904689f589c939c, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.667293 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1714c2a-e7d8-4bba-bc3d-383d4a0aa87c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.677494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1366/2735] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:44 2022] PUT /identity/v3/projects/9d7a20086cdb4bfaa67e44d21691b95a/users/a6f88dc374e6446a8904689f589c939c/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:44.688687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e144da4-7c1a-4282-9dd6-ec21d2dbceb1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.689204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e144da4-7c1a-4282-9dd6-ec21d2dbceb1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.689316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e144da4-7c1a-4282-9dd6-ec21d2dbceb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.721486 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6e144da4-7c1a-4282-9dd6-ec21d2dbceb1 None None] MFA Rules not processed for user `a6f88dc374e6446a8904689f589c939c`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:44.722888 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6e144da4-7c1a-4282-9dd6-ec21d2dbceb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:44.797383 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1370/2736] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:44.809573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1714c2a-e7d8-4bba-bc3d-383d4a0aa87c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:44.811295 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e1714c2a-e7d8-4bba-bc3d-383d4a0aa87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:44.835169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:44.893564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1714c2a-e7d8-4bba-bc3d-383d4a0aa87c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:44.897437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398f6f9-35a0-4f7a-a2f7-502adb1f7af9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:44.898055 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398f6f9-35a0-4f7a-a2f7-502adb1f7af9 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:44.898332 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398f6f9-35a0-4f7a-a2f7-502adb1f7af9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:44.898599 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398f6f9-35a0-4f7a-a2f7-502adb1f7af9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:44.908122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398f6f9-35a0-4f7a-a2f7-502adb1f7af9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:44.909536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398f6f9-35a0-4f7a-a2f7-502adb1f7af9 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:44.918746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1367/2737] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:45.220855 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e144da4-7c1a-4282-9dd6-ec21d2dbceb1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:45.230149 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e144da4-7c1a-4282-9dd6-ec21d2dbceb1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:45.234826 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531953e-ad4f-4de8-b2ae-f055fe463d18 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f88dc374e6446a8904689f589c939c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d7a20086cdb4bfaa67e44d21691b95a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:45.235414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531953e-ad4f-4de8-b2ae-f055fe463d18 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:45.235692 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531953e-ad4f-4de8-b2ae-f055fe463d18 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:45.236229 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531953e-ad4f-4de8-b2ae-f055fe463d18 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:45.236861 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531953e-ad4f-4de8-b2ae-f055fe463d18 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:45.237162 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531953e-ad4f-4de8-b2ae-f055fe463d18 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:45.238784 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531953e-ad4f-4de8-b2ae-f055fe463d18 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:45.250401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1371/2738] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:45.254551 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398f6f9-35a0-4f7a-a2f7-502adb1f7af9 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:45.264099 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398f6f9-35a0-4f7a-a2f7-502adb1f7af9 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:45.268739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bfb78-8a9a-4b9b-90c8-869c2d3e05d4 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f88dc374e6446a8904689f589c939c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d7a20086cdb4bfaa67e44d21691b95a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:45.269333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bfb78-8a9a-4b9b-90c8-869c2d3e05d4 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:45.269616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bfb78-8a9a-4b9b-90c8-869c2d3e05d4 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:45.269892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bfb78-8a9a-4b9b-90c8-869c2d3e05d4 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:45.270577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bfb78-8a9a-4b9b-90c8-869c2d3e05d4 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:45.272444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bfb78-8a9a-4b9b-90c8-869c2d3e05d4 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:45.301957 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1368/2739] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:45 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:45.306087 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531953e-ad4f-4de8-b2ae-f055fe463d18 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:45.317362 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531953e-ad4f-4de8-b2ae-f055fe463d18 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:45.320889 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:45.375633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f88dc374e6446a8904689f589c939c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d7a20086cdb4bfaa67e44d21691b95a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:45.376230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:45.376512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:45.376785 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:45.377413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:45.377714 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:45.379313 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:45.391846 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1372/2740] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:45.428565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bfb78-8a9a-4b9b-90c8-869c2d3e05d4 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:45.439262 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bfb78-8a9a-4b9b-90c8-869c2d3e05d4 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:45.443739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f88dc374e6446a8904689f589c939c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d7a20086cdb4bfaa67e44d21691b95a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:45.444354 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:45.444634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:45.444905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] PATH_INFO: `/v3/projects/c80f79ce989846128fc1898c28ebd246` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:45.446041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c80f79ce989846128fc1898c28ebd246)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:45.447951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:45.492899 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80f79ce989846128fc1898c28ebd246', 'request_id': 'req-126199e7-15b6-4bac-a6b1-dc0797a8053c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.498953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80f79ce989846128fc1898c28ebd246', 'request_id': 'req-126199e7-15b6-4bac-a6b1-dc0797a8053c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.506114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80f79ce989846128fc1898c28ebd246', 'request_id': 'req-126199e7-15b6-4bac-a6b1-dc0797a8053c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.518715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] The token cache is being invalidate because project c80f79ce989846128fc1898c28ebd246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:45.519005 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.519640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.521083 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1369/2741] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:45 2022] DELETE /identity/v3/projects/c80f79ce989846128fc1898c28ebd246 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:45.709580 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:45.711293 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:45.778197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e98eb-e080-4abd-bd94-84108fe43d67 tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:45.786708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:45.788350 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:45.833454 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:45.834115 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:45.834394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:45.834661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] PATH_INFO: `/v3/users/5df88dc067484b1c85e6e6081294df14` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:45.835767 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df88dc067484b1c85e6e6081294df14)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:45.837650 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:45.852376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126199e7-15b6-4bac-a6b1-dc0797a8053c tempest-VolumeQuotaClassesTest-929049322 tempest-VolumeQuotaClassesTest-929049322-project-admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:45.858065 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:45.858851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:45.859192 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:45.859557 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] PATH_INFO: `/v3/users/5fa2031d5e8147b78acd2fd17dc69bf8` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:45.860773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa2031d5e8147b78acd2fd17dc69bf8)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:45.863035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:45.868259 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:45.883868 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df88dc067484b1c85e6e6081294df14'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.886776 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:45.892555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df88dc067484b1c85e6e6081294df14'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.893341 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df88dc067484b1c85e6e6081294df14'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.894064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df88dc067484b1c85e6e6081294df14'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.903609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a1dc7056-b24d-47ca-bf41-a94bcda1b28e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df88dc067484b1c85e6e6081294df14'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.912442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa2031d5e8147b78acd2fd17dc69bf8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.913176 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa2031d5e8147b78acd2fd17dc69bf8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.921373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa2031d5e8147b78acd2fd17dc69bf8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.921905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1373/2742] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:45 2022] DELETE /identity/v3/users/5df88dc067484b1c85e6e6081294df14 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:45.922124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa2031d5e8147b78acd2fd17dc69bf8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.926286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a94a3-62af-4082-8751-71f34dc4253c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:45.926653 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a94a3-62af-4082-8751-71f34dc4253c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:45.926919 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a94a3-62af-4082-8751-71f34dc4253c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:45.929548 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-db1669f6-8a9a-4a1b-b84f-99fbeba521f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa2031d5e8147b78acd2fd17dc69bf8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:45.946351 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5c6a94a3-62af-4082-8751-71f34dc4253c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:45.947375 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1370/2743] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:45 2022] DELETE /identity/v3/users/5fa2031d5e8147b78acd2fd17dc69bf8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:45.950474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5c6a94a3-62af-4082-8751-71f34dc4253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:45.951805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253993-e871-43e4-8ea9-d5591e36de65 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:45.952126 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253993-e871-43e4-8ea9-d5591e36de65 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:45.952401 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253993-e871-43e4-8ea9-d5591e36de65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:45.972057 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-59253993-e871-43e4-8ea9-d5591e36de65 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:45.973330 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-59253993-e871-43e4-8ea9-d5591e36de65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:46.008905 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1374/2744] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:46.018682 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a94a3-62af-4082-8751-71f34dc4253c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:46.020015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5c6a94a3-62af-4082-8751-71f34dc4253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:46.032905 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1371/2745] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:46.043363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59253993-e871-43e4-8ea9-d5591e36de65 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:46.044740 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-59253993-e871-43e4-8ea9-d5591e36de65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:46.079587 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a94a3-62af-4082-8751-71f34dc4253c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:46.083325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d060a31-1a47-4ceb-9ece-c2fa31422623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:46.083996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d060a31-1a47-4ceb-9ece-c2fa31422623 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.084370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d060a31-1a47-4ceb-9ece-c2fa31422623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.084709 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d060a31-1a47-4ceb-9ece-c2fa31422623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.095442 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d060a31-1a47-4ceb-9ece-c2fa31422623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:46.096865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d060a31-1a47-4ceb-9ece-c2fa31422623 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:46.103429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59253993-e871-43e4-8ea9-d5591e36de65 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:46.106355 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1375/2746] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:35:46.107266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ad94e-8300-4a7d-837e-5ba143288c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:46.107873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ad94e-8300-4a7d-837e-5ba143288c05 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.108155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ad94e-8300-4a7d-837e-5ba143288c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.108426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ad94e-8300-4a7d-837e-5ba143288c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.117840 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ad94e-8300-4a7d-837e-5ba143288c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:46.119296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ad94e-8300-4a7d-837e-5ba143288c05 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:46.129420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1372/2747] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:46.483777 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d060a31-1a47-4ceb-9ece-c2fa31422623 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:46.485477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4d060a31-1a47-4ceb-9ece-c2fa31422623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:46.493219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ad94e-8300-4a7d-837e-5ba143288c05 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:46.494603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-802ad94e-8300-4a7d-837e-5ba143288c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:46.504222 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d060a31-1a47-4ceb-9ece-c2fa31422623 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:46.510120 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ad94e-8300-4a7d-837e-5ba143288c05 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:46.510338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:46.511317 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.511665 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.511854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] PATH_INFO: `/v3/projects/b0b0960fc65f440abb014139999dd7ad` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.513007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b0960fc65f440abb014139999dd7ad)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:46.515009 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:46.515135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:46.515752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.516029 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.516302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] PATH_INFO: `/v3/projects/52bd875c1399477bbe5a9fa08d8176f7` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.517524 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=52bd875c1399477bbe5a9fa08d8176f7)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:46.519463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:46.545272 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:46.573697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52bd875c1399477bbe5a9fa08d8176f7', 'request_id': 'req-4949ddf4-dbca-4cff-8253-08d76c7890fe'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.575221 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b0960fc65f440abb014139999dd7ad', 'request_id': 'req-475a6131-a7bd-4927-b9cd-716ab35112a3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.579766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52bd875c1399477bbe5a9fa08d8176f7', 'request_id': 'req-4949ddf4-dbca-4cff-8253-08d76c7890fe'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.582780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b0960fc65f440abb014139999dd7ad', 'request_id': 'req-475a6131-a7bd-4927-b9cd-716ab35112a3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.587110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52bd875c1399477bbe5a9fa08d8176f7', 'request_id': 'req-4949ddf4-dbca-4cff-8253-08d76c7890fe'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.599743 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] The token cache is being invalidate because project 52bd875c1399477bbe5a9fa08d8176f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:46.600035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.600676 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4949ddf4-dbca-4cff-8253-08d76c7890fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.601406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b0960fc65f440abb014139999dd7ad', 'request_id': 'req-475a6131-a7bd-4927-b9cd-716ab35112a3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.602222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1373/2748] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:46 2022] DELETE /identity/v3/projects/52bd875c1399477bbe5a9fa08d8176f7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:46.605833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:46.607167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04026c9-4a75-4e92-ad30-2bda91b0775f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.607553 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04026c9-4a75-4e92-ad30-2bda91b0775f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.607860 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04026c9-4a75-4e92-ad30-2bda91b0775f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.614991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] The token cache is being invalidate because project b0b0960fc65f440abb014139999dd7ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:46.615290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.616180 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.618122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1376/2749] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:46 2022] DELETE /identity/v3/projects/b0b0960fc65f440abb014139999dd7ad => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:46.622206 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:46.623711 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:46.632223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c04026c9-4a75-4e92-ad30-2bda91b0775f None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:46.633495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c04026c9-4a75-4e92-ad30-2bda91b0775f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:46.692741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a6131-a7bd-4927-b9cd-716ab35112a3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:46.695979 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1374/2750] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:46.697406 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:46.698671 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.698953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.699249 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] PATH_INFO: `/v3/users/a6f88dc374e6446a8904689f589c939c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.700413 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f88dc374e6446a8904689f589c939c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:46.702376 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:46.704254 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafbd37-370b-44a0-9ea3-91dd3a26c067 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.704546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafbd37-370b-44a0-9ea3-91dd3a26c067 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.704824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafbd37-370b-44a0-9ea3-91dd3a26c067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.723701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5bafbd37-370b-44a0-9ea3-91dd3a26c067 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:46.724896 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5bafbd37-370b-44a0-9ea3-91dd3a26c067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:46.734478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1375/2751] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:46.738786 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafbd37-370b-44a0-9ea3-91dd3a26c067 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:46.748327 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafbd37-370b-44a0-9ea3-91dd3a26c067 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:46.749845 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f88dc374e6446a8904689f589c939c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.753485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:46.754125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.754410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.754687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.755328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:46.755625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:46.757312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:46.757876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f88dc374e6446a8904689f589c939c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.758546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f88dc374e6446a8904689f589c939c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.759170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f88dc374e6446a8904689f589c939c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.766972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f88dc374e6446a8904689f589c939c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:46.769428 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1376/2752] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:35:46.773574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:46.775233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:46.784204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1377/2753] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:46 2022] DELETE /identity/v3/users/a6f88dc374e6446a8904689f589c939c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:46.787518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5fa9e-b69b-45fa-9115-187fd0630f9d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:46.844895 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1814216-8dd3-4626-a33b-7f3032e9d569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:46.845714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814216-8dd3-4626-a33b-7f3032e9d569 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.846129 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814216-8dd3-4626-a33b-7f3032e9d569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.846495 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814216-8dd3-4626-a33b-7f3032e9d569 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.847201 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1814216-8dd3-4626-a33b-7f3032e9d569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:46.849188 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1814216-8dd3-4626-a33b-7f3032e9d569 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:46.877805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1377/2754] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:46 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:35:46.882133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:46.893083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec215bf-2c63-44cb-9378-c743e447f43a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:46.955026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64615c59-a368-4b37-b931-4116b7b7341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:46.955727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615c59-a368-4b37-b931-4116b7b7341d admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:46.956086 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615c59-a368-4b37-b931-4116b7b7341d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:46.956434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615c59-a368-4b37-b931-4116b7b7341d admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:46.957148 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64615c59-a368-4b37-b931-4116b7b7341d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:46.959167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64615c59-a368-4b37-b931-4116b7b7341d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:46.980970 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1378/2755] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:46 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:35:46.985220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1814216-8dd3-4626-a33b-7f3032e9d569 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:46.996393 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1814216-8dd3-4626-a33b-7f3032e9d569 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.001032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df0fc1-eef9-46fc-b96a-14f4a813bd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.001634 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df0fc1-eef9-46fc-b96a-14f4a813bd5a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.001916 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df0fc1-eef9-46fc-b96a-14f4a813bd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.002218 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df0fc1-eef9-46fc-b96a-14f4a813bd5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.002859 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df0fc1-eef9-46fc-b96a-14f4a813bd5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.004451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df0fc1-eef9-46fc-b96a-14f4a813bd5a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.014411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1378/2756] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:47.018374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64615c59-a368-4b37-b931-4116b7b7341d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.023478 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:47.029890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64615c59-a368-4b37-b931-4116b7b7341d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.035614 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee3f91-1580-400b-9d41-849e4dde11d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.036316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee3f91-1580-400b-9d41-849e4dde11d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.036667 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee3f91-1580-400b-9d41-849e4dde11d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.037014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee3f91-1580-400b-9d41-849e4dde11d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.037730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee3f91-1580-400b-9d41-849e4dde11d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.039419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee3f91-1580-400b-9d41-849e4dde11d4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.050134 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1379/2757] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:47.054340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df0fc1-eef9-46fc-b96a-14f4a813bd5a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.061771 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:47.065969 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df0fc1-eef9-46fc-b96a-14f4a813bd5a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.071001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee3f91-1580-400b-9d41-849e4dde11d4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.072351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0b679-5756-4d9f-9473-ffae0250e8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.073096 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0b679-5756-4d9f-9473-ffae0250e8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.073512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-15ee3f91-1580-400b-9d41-849e4dde11d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:47.073649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0b679-5756-4d9f-9473-ffae0250e8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.074004 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0b679-5756-4d9f-9473-ffae0250e8bf admin admin] PATH_INFO: `/v3/projects/edecb1d4976d4c8994c9a1b5b4d0ca84/users/c260bd45a2474d1889b5d5da5682574f/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.089164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee3f91-1580-400b-9d41-849e4dde11d4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.093141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0b679-5756-4d9f-9473-ffae0250e8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=edecb1d4976d4c8994c9a1b5b4d0ca84, user_id=c260bd45a2474d1889b5d5da5682574f, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.094375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.095014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.095582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.095713 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] PATH_INFO: `/v3/projects/9d7a20086cdb4bfaa67e44d21691b95a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.096892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0b679-5756-4d9f-9473-ffae0250e8bf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.097020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d7a20086cdb4bfaa67e44d21691b95a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.099291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.111820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1379/2758] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:47 2022] PUT /identity/v3/projects/edecb1d4976d4c8994c9a1b5b4d0ca84/users/c260bd45a2474d1889b5d5da5682574f/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:47.122644 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb82b5-8efb-4564-abc2-589ad67ba2ee None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.122931 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb82b5-8efb-4564-abc2-589ad67ba2ee None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.123202 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb82b5-8efb-4564-abc2-589ad67ba2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.152002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f7bb82b5-8efb-4564-abc2-589ad67ba2ee None None] MFA Rules not processed for user `c260bd45a2474d1889b5d5da5682574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:47.153245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f7bb82b5-8efb-4564-abc2-589ad67ba2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:47.155236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d7a20086cdb4bfaa67e44d21691b95a', 'request_id': 'req-350ab455-3dd0-4e87-b30d-20b9e848b542'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:47.163218 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d7a20086cdb4bfaa67e44d21691b95a', 'request_id': 'req-350ab455-3dd0-4e87-b30d-20b9e848b542'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:47.170106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d7a20086cdb4bfaa67e44d21691b95a', 'request_id': 'req-350ab455-3dd0-4e87-b30d-20b9e848b542'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:47.183565 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] The token cache is being invalidate because project 9d7a20086cdb4bfaa67e44d21691b95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:47.183864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:47.184513 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-350ab455-3dd0-4e87-b30d-20b9e848b542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:47.186054 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1380/2759] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:47 2022] DELETE /identity/v3/projects/9d7a20086cdb4bfaa67e44d21691b95a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:47.190738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6d4ff-b45e-4e4f-aa17-f0a5cec2df23 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.190953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6d4ff-b45e-4e4f-aa17-f0a5cec2df23 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.191220 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6d4ff-b45e-4e4f-aa17-f0a5cec2df23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.211583 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-84a6d4ff-b45e-4e4f-aa17-f0a5cec2df23 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:47.212778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-84a6d4ff-b45e-4e4f-aa17-f0a5cec2df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:47.230127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1380/2760] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:47.240438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb82b5-8efb-4564-abc2-589ad67ba2ee None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.241811 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f7bb82b5-8efb-4564-abc2-589ad67ba2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:47.279597 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1381/2761] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:47.288822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c048b0-dabf-422e-9abe-8da454692ec5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.290278 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c048b0-dabf-422e-9abe-8da454692ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.290641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c048b0-dabf-422e-9abe-8da454692ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.299832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb82b5-8efb-4564-abc2-589ad67ba2ee None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.304154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69dbb58-f4f9-4eae-b407-4c7c73f2410a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.304794 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69dbb58-f4f9-4eae-b407-4c7c73f2410a service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.305074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69dbb58-f4f9-4eae-b407-4c7c73f2410a service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.305353 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69dbb58-f4f9-4eae-b407-4c7c73f2410a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.314241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69dbb58-f4f9-4eae-b407-4c7c73f2410a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.315675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69dbb58-f4f9-4eae-b407-4c7c73f2410a service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.316179 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b6c048b0-dabf-422e-9abe-8da454692ec5 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:47.317414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b6c048b0-dabf-422e-9abe-8da454692ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:47.324534 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1381/2762] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:35:47.326176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1382/2763] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:47.330455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69dbb58-f4f9-4eae-b407-4c7c73f2410a service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.342290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69dbb58-f4f9-4eae-b407-4c7c73f2410a service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.346977 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f8149-1095-4304-8e61-408273c51969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.347563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f8149-1095-4304-8e61-408273c51969 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.347877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f8149-1095-4304-8e61-408273c51969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.348154 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f8149-1095-4304-8e61-408273c51969 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.348788 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f8149-1095-4304-8e61-408273c51969 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:47.349094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f8149-1095-4304-8e61-408273c51969 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.350868 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f8149-1095-4304-8e61-408273c51969 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.363919 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1382/2764] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 05:35:47.368139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c048b0-dabf-422e-9abe-8da454692ec5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.377998 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c048b0-dabf-422e-9abe-8da454692ec5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.384100 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1bfd7-3646-4b67-b22f-b73db574bce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.384697 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1bfd7-3646-4b67-b22f-b73db574bce5 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.384974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1bfd7-3646-4b67-b22f-b73db574bce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.385245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1bfd7-3646-4b67-b22f-b73db574bce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.385901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1bfd7-3646-4b67-b22f-b73db574bce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.387780 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1bfd7-3646-4b67-b22f-b73db574bce5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.415399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1383/2765] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:47 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:47.420339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f8149-1095-4304-8e61-408273c51969 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.435790 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f8149-1095-4304-8e61-408273c51969 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.494577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cfff0-e34c-4a30-955d-735ba79f0464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.495312 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cfff0-e34c-4a30-955d-735ba79f0464 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.495485 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cfff0-e34c-4a30-955d-735ba79f0464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.495758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cfff0-e34c-4a30-955d-735ba79f0464 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.496403 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cfff0-e34c-4a30-955d-735ba79f0464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.498363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cfff0-e34c-4a30-955d-735ba79f0464 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.506398 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:47.521210 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1383/2766] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:47 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:47.525382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1bfd7-3646-4b67-b22f-b73db574bce5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.536556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1bfd7-3646-4b67-b22f-b73db574bce5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.542216 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f088f6c-7a9a-441e-a3fa-ca51916b4438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.543532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f088f6c-7a9a-441e-a3fa-ca51916b4438 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.543532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f088f6c-7a9a-441e-a3fa-ca51916b4438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.543532 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f088f6c-7a9a-441e-a3fa-ca51916b4438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.544040 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f088f6c-7a9a-441e-a3fa-ca51916b4438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.546137 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f088f6c-7a9a-441e-a3fa-ca51916b4438 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.556500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1384/2767] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:35:47.560509 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cfff0-e34c-4a30-955d-735ba79f0464 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.582198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cfff0-e34c-4a30-955d-735ba79f0464 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.582198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4552536-3f0b-45f3-941c-7ee1425f3980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.582198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4552536-3f0b-45f3-941c-7ee1425f3980 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.582198 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4552536-3f0b-45f3-941c-7ee1425f3980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.582577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4552536-3f0b-45f3-941c-7ee1425f3980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.582577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4552536-3f0b-45f3-941c-7ee1425f3980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.594503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4552536-3f0b-45f3-941c-7ee1425f3980 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.612433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1384/2768] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:47.616659 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f088f6c-7a9a-441e-a3fa-ca51916b4438 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.636999 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f088f6c-7a9a-441e-a3fa-ca51916b4438 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.641909 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c247ae-75d9-4a4e-a994-9dafb48e3bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.642689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c247ae-75d9-4a4e-a994-9dafb48e3bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.643035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c247ae-75d9-4a4e-a994-9dafb48e3bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.643374 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c247ae-75d9-4a4e-a994-9dafb48e3bf9 admin admin] PATH_INFO: `/v3/projects/8bc01b963d514b19937df50be8503a5a/users/e83a9e8f7f124b359b9f2d5d535f898a/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.659833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c247ae-75d9-4a4e-a994-9dafb48e3bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc01b963d514b19937df50be8503a5a, user_id=e83a9e8f7f124b359b9f2d5d535f898a, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.662696 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c247ae-75d9-4a4e-a994-9dafb48e3bf9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.676444 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1385/2769] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:47 2022] PUT /identity/v3/projects/8bc01b963d514b19937df50be8503a5a/users/e83a9e8f7f124b359b9f2d5d535f898a/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:47.688080 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171844c-f00d-485d-bfd0-7fc2e3cf8d3d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.688372 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171844c-f00d-485d-bfd0-7fc2e3cf8d3d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.688653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171844c-f00d-485d-bfd0-7fc2e3cf8d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.716391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a171844c-f00d-485d-bfd0-7fc2e3cf8d3d None None] MFA Rules not processed for user `e83a9e8f7f124b359b9f2d5d535f898a`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:47.717605 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a171844c-f00d-485d-bfd0-7fc2e3cf8d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:47.780528 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1385/2770] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:47.784917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c247ae-75d9-4a4e-a994-9dafb48e3bf9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.797235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:47.847708 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c247ae-75d9-4a4e-a994-9dafb48e3bf9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.852854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dee43-9bc7-4e6c-9e1d-1241c61441ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.853455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dee43-9bc7-4e6c-9e1d-1241c61441ce admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.853737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dee43-9bc7-4e6c-9e1d-1241c61441ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.854055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dee43-9bc7-4e6c-9e1d-1241c61441ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.854687 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dee43-9bc7-4e6c-9e1d-1241c61441ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.856590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dee43-9bc7-4e6c-9e1d-1241c61441ce admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.883613 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1386/2771] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:47 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:47.888275 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171844c-f00d-485d-bfd0-7fc2e3cf8d3d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.898725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171844c-f00d-485d-bfd0-7fc2e3cf8d3d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.954610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c41992-4a59-4160-bb9f-c699f7532505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.955213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c41992-4a59-4160-bb9f-c699f7532505 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.955494 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c41992-4a59-4160-bb9f-c699f7532505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.955770 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c41992-4a59-4160-bb9f-c699f7532505 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.956388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c41992-4a59-4160-bb9f-c699f7532505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:47.958321 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c41992-4a59-4160-bb9f-c699f7532505 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:47.978618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1386/2772] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:47 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:47.982621 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dee43-9bc7-4e6c-9e1d-1241c61441ce admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:47.993055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dee43-9bc7-4e6c-9e1d-1241c61441ce admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:47.998142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211faf7-fa94-47bc-80c3-dd4a32f6049b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:47.998734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211faf7-fa94-47bc-80c3-dd4a32f6049b admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:47.999017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211faf7-fa94-47bc-80c3-dd4a32f6049b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:47.999291 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211faf7-fa94-47bc-80c3-dd4a32f6049b admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:47.999921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211faf7-fa94-47bc-80c3-dd4a32f6049b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:48.001504 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211faf7-fa94-47bc-80c3-dd4a32f6049b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:48.012373 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1387/2773] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:48.016230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c41992-4a59-4160-bb9f-c699f7532505 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:48.026142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c41992-4a59-4160-bb9f-c699f7532505 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:48.030832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e34dd5-ccc7-47a2-81b1-15c07fa832da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:48.031418 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e34dd5-ccc7-47a2-81b1-15c07fa832da admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:48.031698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e34dd5-ccc7-47a2-81b1-15c07fa832da admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:48.031974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e34dd5-ccc7-47a2-81b1-15c07fa832da admin admin] PATH_INFO: `/v3/projects/d1b567635e8f44eea69d3ac0007823fd/users/c0eafd7f14f14a31a9dbc7fb00afa726/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:48.046266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e34dd5-ccc7-47a2-81b1-15c07fa832da admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b567635e8f44eea69d3ac0007823fd, user_id=c0eafd7f14f14a31a9dbc7fb00afa726, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:48.048920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e34dd5-ccc7-47a2-81b1-15c07fa832da admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:48.061881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1387/2774] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:48 2022] PUT /identity/v3/projects/d1b567635e8f44eea69d3ac0007823fd/users/c0eafd7f14f14a31a9dbc7fb00afa726/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:48.065651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211faf7-fa94-47bc-80c3-dd4a32f6049b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:48.076169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211faf7-fa94-47bc-80c3-dd4a32f6049b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:48.131492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ca87e-be85-4849-b6a7-b9e79f87061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:48.132097 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ca87e-be85-4849-b6a7-b9e79f87061e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:48.132387 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ca87e-be85-4849-b6a7-b9e79f87061e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:48.132664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ca87e-be85-4849-b6a7-b9e79f87061e admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:48.133294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ca87e-be85-4849-b6a7-b9e79f87061e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:48.134908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ca87e-be85-4849-b6a7-b9e79f87061e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:48.145019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1388/2775] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:48.149278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e34dd5-ccc7-47a2-81b1-15c07fa832da admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:48.159823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e34dd5-ccc7-47a2-81b1-15c07fa832da admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:48.164782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e725a1d-3afd-4b56-99b6-722ab2dfc468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:48.165439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e725a1d-3afd-4b56-99b6-722ab2dfc468 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:48.165723 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e725a1d-3afd-4b56-99b6-722ab2dfc468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:48.166036 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e725a1d-3afd-4b56-99b6-722ab2dfc468 admin admin] PATH_INFO: `/v3/projects/d1b567635e8f44eea69d3ac0007823fd/users/c0eafd7f14f14a31a9dbc7fb00afa726/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:48.167952 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e725a1d-3afd-4b56-99b6-722ab2dfc468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b567635e8f44eea69d3ac0007823fd, user_id=c0eafd7f14f14a31a9dbc7fb00afa726, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:48.170649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e725a1d-3afd-4b56-99b6-722ab2dfc468 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:48.180787 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1388/2776] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:48 2022] PUT /identity/v3/projects/d1b567635e8f44eea69d3ac0007823fd/users/c0eafd7f14f14a31a9dbc7fb00afa726/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:48.191610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc50b47-dc01-403f-a145-a51b4872156e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:48.191904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc50b47-dc01-403f-a145-a51b4872156e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:48.192187 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc50b47-dc01-403f-a145-a51b4872156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:48.219869 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7cc50b47-dc01-403f-a145-a51b4872156e None None] MFA Rules not processed for user `c0eafd7f14f14a31a9dbc7fb00afa726`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:48.221074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7cc50b47-dc01-403f-a145-a51b4872156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:48.287045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1389/2777] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:48.298626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e725a1d-3afd-4b56-99b6-722ab2dfc468 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:48.320048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:48.353831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e725a1d-3afd-4b56-99b6-722ab2dfc468 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:48.358266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a39cbf-55cd-4150-8f73-2bc32236ece8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:48.358888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a39cbf-55cd-4150-8f73-2bc32236ece8 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:48.359169 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a39cbf-55cd-4150-8f73-2bc32236ece8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:48.359443 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a39cbf-55cd-4150-8f73-2bc32236ece8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:48.368795 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a39cbf-55cd-4150-8f73-2bc32236ece8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:48.370832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a39cbf-55cd-4150-8f73-2bc32236ece8 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:48.380268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1389/2778] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:48.926976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc50b47-dc01-403f-a145-a51b4872156e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:48.935389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:48.992392 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc50b47-dc01-403f-a145-a51b4872156e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:48.996964 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:48.997590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:48.997865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:48.998164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] PATH_INFO: `/v3/users/e83a9e8f7f124b359b9f2d5d535f898a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:48.999273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e83a9e8f7f124b359b9f2d5d535f898a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:49.001164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:49.046066 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e83a9e8f7f124b359b9f2d5d535f898a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.053663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e83a9e8f7f124b359b9f2d5d535f898a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.054438 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e83a9e8f7f124b359b9f2d5d535f898a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.055073 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e83a9e8f7f124b359b9f2d5d535f898a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.061996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e83a9e8f7f124b359b9f2d5d535f898a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.079382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1390/2779] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:48 2022] DELETE /identity/v3/users/e83a9e8f7f124b359b9f2d5d535f898a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:49.083879 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e884411-8fa5-4db2-bd4d-653e4a6f1c19 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:49.084351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e884411-8fa5-4db2-bd4d-653e4a6f1c19 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:49.084466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e884411-8fa5-4db2-bd4d-653e4a6f1c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:49.103923 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1e884411-8fa5-4db2-bd4d-653e4a6f1c19 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:49.105130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1e884411-8fa5-4db2-bd4d-653e4a6f1c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:49.161471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1390/2780] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:49.171403 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:49.172805 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:49.228959 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca191940-2143-44e5-8c49-b1d0ee0b7311 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:49.233536 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c616e-e993-4067-9790-c0d8ecf9c2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:49.234171 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c616e-e993-4067-9790-c0d8ecf9c2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:49.234458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c616e-e993-4067-9790-c0d8ecf9c2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:49.234733 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c616e-e993-4067-9790-c0d8ecf9c2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:49.243833 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c616e-e993-4067-9790-c0d8ecf9c2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:49.245257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c616e-e993-4067-9790-c0d8ecf9c2c7 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:49.254853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1391/2781] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:49.518918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e884411-8fa5-4db2-bd4d-653e4a6f1c19 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:49.520371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1e884411-8fa5-4db2-bd4d-653e4a6f1c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:49.532033 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e884411-8fa5-4db2-bd4d-653e4a6f1c19 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:49.536531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:49.537137 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:49.537423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:49.537686 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] PATH_INFO: `/v3/projects/8bc01b963d514b19937df50be8503a5a` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:49.538974 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc01b963d514b19937df50be8503a5a)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:49.540886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:49.587286 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc01b963d514b19937df50be8503a5a', 'request_id': 'req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.597304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc01b963d514b19937df50be8503a5a', 'request_id': 'req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.605106 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc01b963d514b19937df50be8503a5a', 'request_id': 'req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.616562 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] The token cache is being invalidate because project 8bc01b963d514b19937df50be8503a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:49.616849 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.617467 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.618986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1391/2782] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:49 2022] DELETE /identity/v3/projects/8bc01b963d514b19937df50be8503a5a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:49.623589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c616e-e993-4067-9790-c0d8ecf9c2c7 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:49.633108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c616e-e993-4067-9790-c0d8ecf9c2c7 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:49.658820 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:49.690450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:49.691102 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:49.691378 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:49.691648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] PATH_INFO: `/v3/users/c0eafd7f14f14a31a9dbc7fb00afa726` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:49.692739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0eafd7f14f14a31a9dbc7fb00afa726)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:49.694657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:49.739164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0eafd7f14f14a31a9dbc7fb00afa726'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.747019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0eafd7f14f14a31a9dbc7fb00afa726'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.747605 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0eafd7f14f14a31a9dbc7fb00afa726'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.748202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0eafd7f14f14a31a9dbc7fb00afa726'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.755220 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ec84a50e-97f1-4895-af40-06428ad0e39c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0eafd7f14f14a31a9dbc7fb00afa726'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:49.772641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1392/2783] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:49 2022] DELETE /identity/v3/users/c0eafd7f14f14a31a9dbc7fb00afa726 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:50.040687 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:50.042700 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:50.058059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c2fba-d6b2-42f7-ba04-52c3a4e3ed77 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:50.102007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:50.116574 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:50.117197 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.117475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.117745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] PATH_INFO: `/v3/projects/d1b567635e8f44eea69d3ac0007823fd` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.118886 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1b567635e8f44eea69d3ac0007823fd)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:50.121447 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:50.165554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1b567635e8f44eea69d3ac0007823fd', 'request_id': 'req-2322d1ff-243e-4585-b96f-a8fa9b195f10'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:50.171541 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1b567635e8f44eea69d3ac0007823fd', 'request_id': 'req-2322d1ff-243e-4585-b96f-a8fa9b195f10'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:50.178465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1b567635e8f44eea69d3ac0007823fd', 'request_id': 'req-2322d1ff-243e-4585-b96f-a8fa9b195f10'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:50.189705 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] The token cache is being invalidate because project d1b567635e8f44eea69d3ac0007823fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:50.190062 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:50.190698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2322d1ff-243e-4585-b96f-a8fa9b195f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:50.192158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1392/2784] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:50 2022] DELETE /identity/v3/projects/d1b567635e8f44eea69d3ac0007823fd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:50.196984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7fb3-8ef4-4e05-a944-4a1317f1547d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.197436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7fb3-8ef4-4e05-a944-4a1317f1547d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.197549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7fb3-8ef4-4e05-a944-4a1317f1547d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.217112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-dabc7fb3-8ef4-4e05-a944-4a1317f1547d None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:50.218342 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dabc7fb3-8ef4-4e05-a944-4a1317f1547d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:50.275476 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1393/2785] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:50.283815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4a95-a3a0-408d-8385-81b346a5589c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.284095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4a95-a3a0-408d-8385-81b346a5589c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.284364 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4a95-a3a0-408d-8385-81b346a5589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.304006 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ac3e4a95-a3a0-408d-8385-81b346a5589c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:50.305186 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ac3e4a95-a3a0-408d-8385-81b346a5589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:50.312901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1393/2786] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:50.316947 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc7fb3-8ef4-4e05-a944-4a1317f1547d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:50.326528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc7fb3-8ef4-4e05-a944-4a1317f1547d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:50.331118 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682e064-0395-4286-957c-aed088947ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:50.331725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682e064-0395-4286-957c-aed088947ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.332004 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682e064-0395-4286-957c-aed088947ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.332275 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682e064-0395-4286-957c-aed088947ea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.332908 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682e064-0395-4286-957c-aed088947ea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:50.333210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682e064-0395-4286-957c-aed088947ea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:50.334841 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682e064-0395-4286-957c-aed088947ea6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:50.346622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1394/2787] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:50.350796 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e4a95-a3a0-408d-8385-81b346a5589c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:50.360304 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e4a95-a3a0-408d-8385-81b346a5589c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:50.365076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb53c78-b67d-4e9c-b2d1-03330534dee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:50.365662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb53c78-b67d-4e9c-b2d1-03330534dee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.365959 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb53c78-b67d-4e9c-b2d1-03330534dee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.366234 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb53c78-b67d-4e9c-b2d1-03330534dee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.366857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb53c78-b67d-4e9c-b2d1-03330534dee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:50.368712 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb53c78-b67d-4e9c-b2d1-03330534dee5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:50.394563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1394/2788] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:50 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:50.398664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682e064-0395-4286-957c-aed088947ea6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:50.408992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682e064-0395-4286-957c-aed088947ea6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:50.440750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:50.466734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31e0e5-ee4b-4eb9-90b8-b6ba4f2e00b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:50.467325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31e0e5-ee4b-4eb9-90b8-b6ba4f2e00b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.467628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31e0e5-ee4b-4eb9-90b8-b6ba4f2e00b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.467898 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31e0e5-ee4b-4eb9-90b8-b6ba4f2e00b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.468518 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31e0e5-ee4b-4eb9-90b8-b6ba4f2e00b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:50.470425 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31e0e5-ee4b-4eb9-90b8-b6ba4f2e00b6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:50.491655 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1395/2789] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:50 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:35:50.501302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb53c78-b67d-4e9c-b2d1-03330534dee5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:50.511488 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb53c78-b67d-4e9c-b2d1-03330534dee5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:50.516079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c37c7-2253-496d-931b-79bf8bddc209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:50.516668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c37c7-2253-496d-931b-79bf8bddc209 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.516943 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c37c7-2253-496d-931b-79bf8bddc209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.517213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c37c7-2253-496d-931b-79bf8bddc209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.517843 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c37c7-2253-496d-931b-79bf8bddc209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:50.519456 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c37c7-2253-496d-931b-79bf8bddc209 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:50.529640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1395/2790] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:35:50.533626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31e0e5-ee4b-4eb9-90b8-b6ba4f2e00b6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:50.545152 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31e0e5-ee4b-4eb9-90b8-b6ba4f2e00b6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:50.549922 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5ca14-4b8a-44f3-b4ff-41fe18d02802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:50.550558 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5ca14-4b8a-44f3-b4ff-41fe18d02802 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.550864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5ca14-4b8a-44f3-b4ff-41fe18d02802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.551136 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5ca14-4b8a-44f3-b4ff-41fe18d02802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.551770 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5ca14-4b8a-44f3-b4ff-41fe18d02802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:50.553371 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5ca14-4b8a-44f3-b4ff-41fe18d02802 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:50.563333 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1396/2791] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:50.567250 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c37c7-2253-496d-931b-79bf8bddc209 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:50.577434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c37c7-2253-496d-931b-79bf8bddc209 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:50.581912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a500e-f9d4-4012-a6a8-b28254ac2e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:50.582575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a500e-f9d4-4012-a6a8-b28254ac2e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.582853 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a500e-f9d4-4012-a6a8-b28254ac2e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.583124 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a500e-f9d4-4012-a6a8-b28254ac2e49 admin admin] PATH_INFO: `/v3/projects/b8a7601f372e452e86c17cfb1f465c10/users/345e0a3ce1de4bcfabf2dde0e78c4371/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.598448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a500e-f9d4-4012-a6a8-b28254ac2e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a7601f372e452e86c17cfb1f465c10, user_id=345e0a3ce1de4bcfabf2dde0e78c4371, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:50.601085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a500e-f9d4-4012-a6a8-b28254ac2e49 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:50.613755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1396/2792] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:50 2022] PUT /identity/v3/projects/b8a7601f372e452e86c17cfb1f465c10/users/345e0a3ce1de4bcfabf2dde0e78c4371/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:50.676747 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d3588-e240-4efe-acca-e2fc0958f437 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.676969 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d3588-e240-4efe-acca-e2fc0958f437 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.677198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d3588-e240-4efe-acca-e2fc0958f437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.707272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-566d3588-e240-4efe-acca-e2fc0958f437 None None] MFA Rules not processed for user `345e0a3ce1de4bcfabf2dde0e78c4371`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:50.708487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-566d3588-e240-4efe-acca-e2fc0958f437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:50.772601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1397/2793] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:50.784361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a500e-f9d4-4012-a6a8-b28254ac2e49 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:50.786133 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2e0a500e-f9d4-4012-a6a8-b28254ac2e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:50.841694 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a500e-f9d4-4012-a6a8-b28254ac2e49 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:50.845257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59df735-6550-4813-a81f-c188cc701637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:50.845842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59df735-6550-4813-a81f-c188cc701637 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.846143 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59df735-6550-4813-a81f-c188cc701637 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.846411 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59df735-6550-4813-a81f-c188cc701637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.854361 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59df735-6550-4813-a81f-c188cc701637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:50.855757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59df735-6550-4813-a81f-c188cc701637 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:50.864442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1397/2794] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:35:50.886455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d3588-e240-4efe-acca-e2fc0958f437 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:50.949199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d3588-e240-4efe-acca-e2fc0958f437 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:50.953756 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:50.954426 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:50.954710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:50.954990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] PATH_INFO: `/v3/users/345e0a3ce1de4bcfabf2dde0e78c4371` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:50.956127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=345e0a3ce1de4bcfabf2dde0e78c4371)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:50.958073 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:51.001956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:51.003761 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '345e0a3ce1de4bcfabf2dde0e78c4371'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.012013 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '345e0a3ce1de4bcfabf2dde0e78c4371'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.012965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '345e0a3ce1de4bcfabf2dde0e78c4371'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.013607 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '345e0a3ce1de4bcfabf2dde0e78c4371'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.020440 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '345e0a3ce1de4bcfabf2dde0e78c4371'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.037026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1398/2795] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:50 2022] DELETE /identity/v3/users/345e0a3ce1de4bcfabf2dde0e78c4371 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:51.041647 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bf35d-482e-4e34-89de-188cc0d65ce1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.041950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bf35d-482e-4e34-89de-188cc0d65ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.042237 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bf35d-482e-4e34-89de-188cc0d65ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.061109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fe0bf35d-482e-4e34-89de-188cc0d65ce1 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:51.062333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fe0bf35d-482e-4e34-89de-188cc0d65ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:51.071261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:51.120228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1398/2796] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:51.129904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:51.131507 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:51.189359 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c01a8-4c25-4cd7-bf66-ede21d3381fc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:51.193002 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68134a-29aa-4eac-9ecd-c4b34a07fede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:51.193635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68134a-29aa-4eac-9ecd-c4b34a07fede service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.193916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68134a-29aa-4eac-9ecd-c4b34a07fede service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.194224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68134a-29aa-4eac-9ecd-c4b34a07fede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.203927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68134a-29aa-4eac-9ecd-c4b34a07fede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:51.205382 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68134a-29aa-4eac-9ecd-c4b34a07fede service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:51.214779 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1399/2797] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:51.483123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bf35d-482e-4e34-89de-188cc0d65ce1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:51.484604 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fe0bf35d-482e-4e34-89de-188cc0d65ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:51.497906 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bf35d-482e-4e34-89de-188cc0d65ce1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:51.502823 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:51.503487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.503793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.504064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] PATH_INFO: `/v3/projects/b8a7601f372e452e86c17cfb1f465c10` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.505255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a7601f372e452e86c17cfb1f465c10)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:51.507290 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:51.555650 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a7601f372e452e86c17cfb1f465c10', 'request_id': 'req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.562091 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a7601f372e452e86c17cfb1f465c10', 'request_id': 'req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.569697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a7601f372e452e86c17cfb1f465c10', 'request_id': 'req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.580614 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] The token cache is being invalidate because project b8a7601f372e452e86c17cfb1f465c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:51.580904 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.581525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1bac7e3d-aef8-4bd6-bbac-eb7b29188d7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:51.583145 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1399/2798] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:51 2022] DELETE /identity/v3/projects/b8a7601f372e452e86c17cfb1f465c10 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:51.588080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb6f18-1043-4b32-9a0d-b56a614f1f03 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.588429 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb6f18-1043-4b32-9a0d-b56a614f1f03 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.588887 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb6f18-1043-4b32-9a0d-b56a614f1f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.611321 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-dcdb6f18-1043-4b32-9a0d-b56a614f1f03 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:51.612525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dcdb6f18-1043-4b32-9a0d-b56a614f1f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:51.674847 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1400/2799] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:51.682516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129c5c4-1e2f-496b-a528-8ef1dd5f6d4c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.682798 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129c5c4-1e2f-496b-a528-8ef1dd5f6d4c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.683070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129c5c4-1e2f-496b-a528-8ef1dd5f6d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.703257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c129c5c4-1e2f-496b-a528-8ef1dd5f6d4c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:51.704475 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c129c5c4-1e2f-496b-a528-8ef1dd5f6d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:51.712374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1400/2800] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:51.717080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb6f18-1043-4b32-9a0d-b56a614f1f03 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:51.726674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb6f18-1043-4b32-9a0d-b56a614f1f03 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:51.731854 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd940359-d7fe-4a83-8230-b4d894cb2aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:51.732448 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd940359-d7fe-4a83-8230-b4d894cb2aac admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.732731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd940359-d7fe-4a83-8230-b4d894cb2aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.733007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd940359-d7fe-4a83-8230-b4d894cb2aac admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.733646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd940359-d7fe-4a83-8230-b4d894cb2aac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:51.733980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd940359-d7fe-4a83-8230-b4d894cb2aac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:51.735564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd940359-d7fe-4a83-8230-b4d894cb2aac admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:51.747467 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1401/2801] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:51.752520 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129c5c4-1e2f-496b-a528-8ef1dd5f6d4c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:51.761603 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129c5c4-1e2f-496b-a528-8ef1dd5f6d4c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:51.766213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf70c8-9781-445f-9755-4cd8f343ec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:51.766808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf70c8-9781-445f-9755-4cd8f343ec6c admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.767082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf70c8-9781-445f-9755-4cd8f343ec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.767351 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf70c8-9781-445f-9755-4cd8f343ec6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.767964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf70c8-9781-445f-9755-4cd8f343ec6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:51.769833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf70c8-9781-445f-9755-4cd8f343ec6c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:51.796344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1401/2802] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:51 2022] POST /identity/v3/projects => generated 361 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:51.800620 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd940359-d7fe-4a83-8230-b4d894cb2aac admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:51.810497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd940359-d7fe-4a83-8230-b4d894cb2aac admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:51.857793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:51.868739 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc6a0c-5aa1-4ec1-8665-f25b83ee3f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:51.869498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc6a0c-5aa1-4ec1-8665-f25b83ee3f0a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.869976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc6a0c-5aa1-4ec1-8665-f25b83ee3f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.870283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc6a0c-5aa1-4ec1-8665-f25b83ee3f0a admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.871059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc6a0c-5aa1-4ec1-8665-f25b83ee3f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:51.872930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc6a0c-5aa1-4ec1-8665-f25b83ee3f0a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:51.892824 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1402/2803] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:51 2022] POST /identity/v3/users => generated 281 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:51.896964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf70c8-9781-445f-9755-4cd8f343ec6c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:51.907626 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf70c8-9781-445f-9755-4cd8f343ec6c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:51.912279 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06693b3-2d0d-449a-9d44-8a9196edad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:51.912873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06693b3-2d0d-449a-9d44-8a9196edad80 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.913152 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06693b3-2d0d-449a-9d44-8a9196edad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.913425 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06693b3-2d0d-449a-9d44-8a9196edad80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.914090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06693b3-2d0d-449a-9d44-8a9196edad80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:51.915671 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06693b3-2d0d-449a-9d44-8a9196edad80 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:51.926426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1402/2804] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:51.930092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc6a0c-5aa1-4ec1-8665-f25b83ee3f0a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:51.939786 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc6a0c-5aa1-4ec1-8665-f25b83ee3f0a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:51.944282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978697f0-46c4-48b7-92ad-22b36737dcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:51.944866 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978697f0-46c4-48b7-92ad-22b36737dcbc admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.945145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978697f0-46c4-48b7-92ad-22b36737dcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.945419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978697f0-46c4-48b7-92ad-22b36737dcbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.946088 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978697f0-46c4-48b7-92ad-22b36737dcbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:51.947651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978697f0-46c4-48b7-92ad-22b36737dcbc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:51.958124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1403/2805] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:51.962100 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06693b3-2d0d-449a-9d44-8a9196edad80 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:51.972423 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06693b3-2d0d-449a-9d44-8a9196edad80 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:51.977071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb6a68-0c1a-4044-ae2c-739a95161fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:51.977677 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb6a68-0c1a-4044-ae2c-739a95161fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:51.977964 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb6a68-0c1a-4044-ae2c-739a95161fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:51.978241 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb6a68-0c1a-4044-ae2c-739a95161fbc admin admin] PATH_INFO: `/v3/projects/34c174fb366740e8b1c68b9dd50fe2e2/users/7f48b28e7f274f0bafa7806948c1cc91/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:51.996696 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb6a68-0c1a-4044-ae2c-739a95161fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c174fb366740e8b1c68b9dd50fe2e2, user_id=7f48b28e7f274f0bafa7806948c1cc91, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:51.999356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb6a68-0c1a-4044-ae2c-739a95161fbc admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:52.012818 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1403/2806] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:51 2022] PUT /identity/v3/projects/34c174fb366740e8b1c68b9dd50fe2e2/users/7f48b28e7f274f0bafa7806948c1cc91/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:52.025314 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524d5f-fde9-4fec-939b-86ef317f304b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:52.026394 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524d5f-fde9-4fec-939b-86ef317f304b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:52.026902 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524d5f-fde9-4fec-939b-86ef317f304b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:52.057353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5b524d5f-fde9-4fec-939b-86ef317f304b None None] MFA Rules not processed for user `7f48b28e7f274f0bafa7806948c1cc91`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:52.058832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5b524d5f-fde9-4fec-939b-86ef317f304b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:52.130241 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1404/2807] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:52.167405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb6a68-0c1a-4044-ae2c-739a95161fbc admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:52.169127 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-68fb6a68-0c1a-4044-ae2c-739a95161fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:52.208235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:52.227429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb6a68-0c1a-4044-ae2c-739a95161fbc admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:52.230970 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02a6b9-f747-422b-af7f-b76c1d520f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:52.231567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02a6b9-f747-422b-af7f-b76c1d520f19 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:52.231842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02a6b9-f747-422b-af7f-b76c1d520f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:52.232110 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02a6b9-f747-422b-af7f-b76c1d520f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:52.240427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02a6b9-f747-422b-af7f-b76c1d520f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:52.241835 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02a6b9-f747-422b-af7f-b76c1d520f19 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:52.250352 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1404/2808] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:52.266962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524d5f-fde9-4fec-939b-86ef317f304b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:52.328622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524d5f-fde9-4fec-939b-86ef317f304b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:52.333270 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:52.333894 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:52.334204 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:52.334474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] PATH_INFO: `/v3/users/7f48b28e7f274f0bafa7806948c1cc91` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:52.335624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f48b28e7f274f0bafa7806948c1cc91)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:52.337529 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:52.382643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f48b28e7f274f0bafa7806948c1cc91'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.390662 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f48b28e7f274f0bafa7806948c1cc91'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.391460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f48b28e7f274f0bafa7806948c1cc91'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.392074 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f48b28e7f274f0bafa7806948c1cc91'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.398965 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f48b28e7f274f0bafa7806948c1cc91'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.400049 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:52.418738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1405/2809] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:35:52 2022] DELETE /identity/v3/users/7f48b28e7f274f0bafa7806948c1cc91 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:52.423337 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdca06-36b9-4935-9987-7dd3ec9f1660 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:52.423699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdca06-36b9-4935-9987-7dd3ec9f1660 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:52.424044 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdca06-36b9-4935-9987-7dd3ec9f1660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:52.443446 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4bbdca06-36b9-4935-9987-7dd3ec9f1660 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:52.444639 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4bbdca06-36b9-4935-9987-7dd3ec9f1660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:52.503665 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1405/2810] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 05:35:52.513925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:52.515363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:52.570526 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04d41-a7c1-4d4b-becd-ea7575707d33 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:52.574966 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02393ebc-b480-46d5-a1ff-257dfbb34f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:52.575595 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02393ebc-b480-46d5-a1ff-257dfbb34f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:52.575871 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02393ebc-b480-46d5-a1ff-257dfbb34f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:52.576140 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02393ebc-b480-46d5-a1ff-257dfbb34f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:52.585363 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02393ebc-b480-46d5-a1ff-257dfbb34f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:52.587453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02393ebc-b480-46d5-a1ff-257dfbb34f87 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:52.597210 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1406/2811] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:52.855303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdca06-36b9-4935-9987-7dd3ec9f1660 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:52.857274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4bbdca06-36b9-4935-9987-7dd3ec9f1660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:52.870289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdca06-36b9-4935-9987-7dd3ec9f1660 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:52.874800 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:52.875421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:52.875695 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:52.875962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] PATH_INFO: `/v3/projects/34c174fb366740e8b1c68b9dd50fe2e2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:52.877067 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c174fb366740e8b1c68b9dd50fe2e2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:52.878986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:52.924265 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c174fb366740e8b1c68b9dd50fe2e2', 'request_id': 'req-499169cb-3d6e-441e-ae77-553bdbf5c942'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.929651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c174fb366740e8b1c68b9dd50fe2e2', 'request_id': 'req-499169cb-3d6e-441e-ae77-553bdbf5c942'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.937813 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c174fb366740e8b1c68b9dd50fe2e2', 'request_id': 'req-499169cb-3d6e-441e-ae77-553bdbf5c942'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.948701 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] The token cache is being invalidate because project 34c174fb366740e8b1c68b9dd50fe2e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:35:52.948991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.949611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-499169cb-3d6e-441e-ae77-553bdbf5c942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:35:52.951090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1406/2812] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:35:52 2022] DELETE /identity/v3/projects/34c174fb366740e8b1c68b9dd50fe2e2 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:52.955945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261dba8-69f8-4241-a5de-3d89ab8d0bc2 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:52.956240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261dba8-69f8-4241-a5de-3d89ab8d0bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:52.956525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261dba8-69f8-4241-a5de-3d89ab8d0bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:52.975706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a261dba8-69f8-4241-a5de-3d89ab8d0bc2 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:52.976897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a261dba8-69f8-4241-a5de-3d89ab8d0bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:53.034957 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1407/2813] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:53.043101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4de13-9541-4ac2-840e-06a2a43d4cf7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.043384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4de13-9541-4ac2-840e-06a2a43d4cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.043655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4de13-9541-4ac2-840e-06a2a43d4cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.063366 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f1d4de13-9541-4ac2-840e-06a2a43d4cf7 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:53.064539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f1d4de13-9541-4ac2-840e-06a2a43d4cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:53.072200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1407/2814] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:53.076212 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261dba8-69f8-4241-a5de-3d89ab8d0bc2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.085746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261dba8-69f8-4241-a5de-3d89ab8d0bc2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.090280 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5857e-af88-4799-bc47-4b29320f9b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.090867 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5857e-af88-4799-bc47-4b29320f9b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.091146 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5857e-af88-4799-bc47-4b29320f9b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.091414 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5857e-af88-4799-bc47-4b29320f9b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.092045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5857e-af88-4799-bc47-4b29320f9b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:35:53.092343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5857e-af88-4799-bc47-4b29320f9b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.093897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5857e-af88-4799-bc47-4b29320f9b43 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.105492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1408/2815] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:35:53.109433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4de13-9541-4ac2-840e-06a2a43d4cf7 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.118601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4de13-9541-4ac2-840e-06a2a43d4cf7 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.123730 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f289fb8-4080-4af2-a0bd-f4fdb9cca433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.124303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f289fb8-4080-4af2-a0bd-f4fdb9cca433 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.124576 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f289fb8-4080-4af2-a0bd-f4fdb9cca433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.124842 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f289fb8-4080-4af2-a0bd-f4fdb9cca433 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.125455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f289fb8-4080-4af2-a0bd-f4fdb9cca433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.127374 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f289fb8-4080-4af2-a0bd-f4fdb9cca433 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.156571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1408/2816] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:53 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:35:53.157956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5857e-af88-4799-bc47-4b29320f9b43 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.168943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5857e-af88-4799-bc47-4b29320f9b43 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.230707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bc870-a2f1-4327-88e0-8513bf250322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.231320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bc870-a2f1-4327-88e0-8513bf250322 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.231603 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bc870-a2f1-4327-88e0-8513bf250322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.231877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bc870-a2f1-4327-88e0-8513bf250322 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.232500 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bc870-a2f1-4327-88e0-8513bf250322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.234439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bc870-a2f1-4327-88e0-8513bf250322 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.250788 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:53.256591 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1409/2817] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:53 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:53.260527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f289fb8-4080-4af2-a0bd-f4fdb9cca433 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.271506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f289fb8-4080-4af2-a0bd-f4fdb9cca433 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.276479 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debf9e0-0449-445c-9cb6-f7b2932ce66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.277066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debf9e0-0449-445c-9cb6-f7b2932ce66c admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.277347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debf9e0-0449-445c-9cb6-f7b2932ce66c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.277610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debf9e0-0449-445c-9cb6-f7b2932ce66c admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.278271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debf9e0-0449-445c-9cb6-f7b2932ce66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.279834 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debf9e0-0449-445c-9cb6-f7b2932ce66c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.289481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1409/2818] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:53.293874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bc870-a2f1-4327-88e0-8513bf250322 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.303474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bc870-a2f1-4327-88e0-8513bf250322 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.307967 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e5dd1-e789-4773-980e-6d0254724abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.308552 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e5dd1-e789-4773-980e-6d0254724abc admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.308828 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e5dd1-e789-4773-980e-6d0254724abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.309139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e5dd1-e789-4773-980e-6d0254724abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.309768 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e5dd1-e789-4773-980e-6d0254724abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.311357 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e5dd1-e789-4773-980e-6d0254724abc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.321741 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1410/2819] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:53.325616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debf9e0-0449-445c-9cb6-f7b2932ce66c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.335183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debf9e0-0449-445c-9cb6-f7b2932ce66c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.340418 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908650d3-9bbf-47e5-afdd-6e04d80701f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.341061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908650d3-9bbf-47e5-afdd-6e04d80701f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.341339 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908650d3-9bbf-47e5-afdd-6e04d80701f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.341608 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908650d3-9bbf-47e5-afdd-6e04d80701f0 admin admin] PATH_INFO: `/v3/projects/92622c92d2aa423eaf4d4baf4a932423/users/e4bf3b3c35c14e15ba0702eee03a61df/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.358063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908650d3-9bbf-47e5-afdd-6e04d80701f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92622c92d2aa423eaf4d4baf4a932423, user_id=e4bf3b3c35c14e15ba0702eee03a61df, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.360708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908650d3-9bbf-47e5-afdd-6e04d80701f0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.368471 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:53.375248 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1410/2820] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:53 2022] PUT /identity/v3/projects/92622c92d2aa423eaf4d4baf4a932423/users/e4bf3b3c35c14e15ba0702eee03a61df/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:53.385707 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc65569-c2a9-468d-bf4b-b15380da26ed None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.386023 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc65569-c2a9-468d-bf4b-b15380da26ed None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.386301 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc65569-c2a9-468d-bf4b-b15380da26ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.413514 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6bc65569-c2a9-468d-bf4b-b15380da26ed None None] MFA Rules not processed for user `e4bf3b3c35c14e15ba0702eee03a61df`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:53.414740 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6bc65569-c2a9-468d-bf4b-b15380da26ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:53.476963 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1411/2821] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:53.482233 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908650d3-9bbf-47e5-afdd-6e04d80701f0 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.542144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908650d3-9bbf-47e5-afdd-6e04d80701f0 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.546750 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370adc51-5dde-4b7f-bd55-d914ee9084ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.547347 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370adc51-5dde-4b7f-bd55-d914ee9084ea admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.547625 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370adc51-5dde-4b7f-bd55-d914ee9084ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.547898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370adc51-5dde-4b7f-bd55-d914ee9084ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.548529 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370adc51-5dde-4b7f-bd55-d914ee9084ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.550455 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370adc51-5dde-4b7f-bd55-d914ee9084ea admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.576500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1411/2822] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:53 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:53.580492 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc65569-c2a9-468d-bf4b-b15380da26ed None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.593745 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc65569-c2a9-468d-bf4b-b15380da26ed None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.650080 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871f15a-f141-4405-a8e2-08456319cf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.650684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871f15a-f141-4405-a8e2-08456319cf44 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.651567 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871f15a-f141-4405-a8e2-08456319cf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.651925 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871f15a-f141-4405-a8e2-08456319cf44 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.652569 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871f15a-f141-4405-a8e2-08456319cf44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.654494 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871f15a-f141-4405-a8e2-08456319cf44 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.675332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1412/2823] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:53 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:53.679376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370adc51-5dde-4b7f-bd55-d914ee9084ea admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.689522 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370adc51-5dde-4b7f-bd55-d914ee9084ea admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.694104 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453b739-adbc-4e2c-bec2-f7b81fb37824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.694690 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453b739-adbc-4e2c-bec2-f7b81fb37824 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.694967 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453b739-adbc-4e2c-bec2-f7b81fb37824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.695235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453b739-adbc-4e2c-bec2-f7b81fb37824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.695863 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453b739-adbc-4e2c-bec2-f7b81fb37824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.697463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453b739-adbc-4e2c-bec2-f7b81fb37824 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.708245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1412/2824] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:53.711910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871f15a-f141-4405-a8e2-08456319cf44 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.722316 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871f15a-f141-4405-a8e2-08456319cf44 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.726886 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b0f88-4c7a-41b8-ab20-8430d39f533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.727474 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b0f88-4c7a-41b8-ab20-8430d39f533e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.727757 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b0f88-4c7a-41b8-ab20-8430d39f533e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.728033 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b0f88-4c7a-41b8-ab20-8430d39f533e admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.728849 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b0f88-4c7a-41b8-ab20-8430d39f533e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.730464 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b0f88-4c7a-41b8-ab20-8430d39f533e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.740484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1413/2825] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:53.744420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453b739-adbc-4e2c-bec2-f7b81fb37824 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.754223 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453b739-adbc-4e2c-bec2-f7b81fb37824 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.758767 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b04fed-5fcf-4f65-95c0-9153421b6da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.759500 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b04fed-5fcf-4f65-95c0-9153421b6da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.759782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b04fed-5fcf-4f65-95c0-9153421b6da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.760058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b04fed-5fcf-4f65-95c0-9153421b6da4 admin admin] PATH_INFO: `/v3/projects/2766dc8d1815456b9118e9bbbab9a6fe/users/83022be705a4458b8fed7ae5d5412056/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.776581 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b04fed-5fcf-4f65-95c0-9153421b6da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2766dc8d1815456b9118e9bbbab9a6fe, user_id=83022be705a4458b8fed7ae5d5412056, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.779230 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b04fed-5fcf-4f65-95c0-9153421b6da4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:53.791453 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1413/2826] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:53 2022] PUT /identity/v3/projects/2766dc8d1815456b9118e9bbbab9a6fe/users/83022be705a4458b8fed7ae5d5412056/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:53.802797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957a064-c1ee-49f2-befd-82f97e09e8e8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.803026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957a064-c1ee-49f2-befd-82f97e09e8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.803329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957a064-c1ee-49f2-befd-82f97e09e8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.831031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f957a064-c1ee-49f2-befd-82f97e09e8e8 None None] MFA Rules not processed for user `83022be705a4458b8fed7ae5d5412056`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:53.832230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f957a064-c1ee-49f2-befd-82f97e09e8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:53.875746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:53.895917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1414/2827] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:53.900433 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b04fed-5fcf-4f65-95c0-9153421b6da4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:53.962135 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b04fed-5fcf-4f65-95c0-9153421b6da4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:53.967873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46975401-f89e-4c00-a8aa-e5130ba780b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:53.968917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46975401-f89e-4c00-a8aa-e5130ba780b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:53.969434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46975401-f89e-4c00-a8aa-e5130ba780b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:53.969951 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46975401-f89e-4c00-a8aa-e5130ba780b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:53.970765 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46975401-f89e-4c00-a8aa-e5130ba780b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:53.972957 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46975401-f89e-4c00-a8aa-e5130ba780b3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:54.000803 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1414/2828] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:35:53 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:54.005158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957a064-c1ee-49f2-befd-82f97e09e8e8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:54.016294 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957a064-c1ee-49f2-befd-82f97e09e8e8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:54.071124 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e455c6-a4b5-4c08-90f4-e0485a29fe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:54.071656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e455c6-a4b5-4c08-90f4-e0485a29fe35 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:54.071939 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e455c6-a4b5-4c08-90f4-e0485a29fe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:54.072214 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e455c6-a4b5-4c08-90f4-e0485a29fe35 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:54.073484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e455c6-a4b5-4c08-90f4-e0485a29fe35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:54.075412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e455c6-a4b5-4c08-90f4-e0485a29fe35 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:54.095980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1415/2829] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:35:54 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:35:54.099932 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46975401-f89e-4c00-a8aa-e5130ba780b3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:54.109890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46975401-f89e-4c00-a8aa-e5130ba780b3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:54.114717 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986af13-f352-4118-8fd2-902d6814a318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:54.115298 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986af13-f352-4118-8fd2-902d6814a318 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:54.115575 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986af13-f352-4118-8fd2-902d6814a318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:54.115848 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986af13-f352-4118-8fd2-902d6814a318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:54.116470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986af13-f352-4118-8fd2-902d6814a318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:54.118076 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986af13-f352-4118-8fd2-902d6814a318 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:54.128720 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1415/2830] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:54.132718 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e455c6-a4b5-4c08-90f4-e0485a29fe35 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:54.143131 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e455c6-a4b5-4c08-90f4-e0485a29fe35 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:54.147879 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7c7e5-914a-4817-822a-8f70bc906f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:54.148515 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7c7e5-914a-4817-822a-8f70bc906f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:54.148795 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7c7e5-914a-4817-822a-8f70bc906f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:54.149071 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7c7e5-914a-4817-822a-8f70bc906f3f admin admin] PATH_INFO: `/v3/projects/bd593bc662964f8da5c990f1b7e04263/users/3a62c5de497f49ea90c339ec3b6c511b/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:54.163546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7c7e5-914a-4817-822a-8f70bc906f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd593bc662964f8da5c990f1b7e04263, user_id=3a62c5de497f49ea90c339ec3b6c511b, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:54.166325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7c7e5-914a-4817-822a-8f70bc906f3f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:54.179585 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1416/2831] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:54 2022] PUT /identity/v3/projects/bd593bc662964f8da5c990f1b7e04263/users/3a62c5de497f49ea90c339ec3b6c511b/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:35:54.184063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986af13-f352-4118-8fd2-902d6814a318 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:54.194593 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986af13-f352-4118-8fd2-902d6814a318 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:54.213588 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:54.251921 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e85ea-627c-45d0-9ba2-5f91af3fe441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:54.252525 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e85ea-627c-45d0-9ba2-5f91af3fe441 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:54.252801 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e85ea-627c-45d0-9ba2-5f91af3fe441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:54.253071 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e85ea-627c-45d0-9ba2-5f91af3fe441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:54.253698 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e85ea-627c-45d0-9ba2-5f91af3fe441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:54.255316 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e85ea-627c-45d0-9ba2-5f91af3fe441 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:54.265429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1416/2832] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:35:54.269880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7c7e5-914a-4817-822a-8f70bc906f3f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:54.280159 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7c7e5-914a-4817-822a-8f70bc906f3f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:54.284722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4ff89-bb54-4e75-bf91-9243474fc4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:54.285375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4ff89-bb54-4e75-bf91-9243474fc4a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:54.285656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4ff89-bb54-4e75-bf91-9243474fc4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:54.285928 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4ff89-bb54-4e75-bf91-9243474fc4a2 admin admin] PATH_INFO: `/v3/projects/bd593bc662964f8da5c990f1b7e04263/users/3a62c5de497f49ea90c339ec3b6c511b/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:54.287926 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4ff89-bb54-4e75-bf91-9243474fc4a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd593bc662964f8da5c990f1b7e04263, user_id=3a62c5de497f49ea90c339ec3b6c511b, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:54.290639 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4ff89-bb54-4e75-bf91-9243474fc4a2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:54.301200 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1417/2833] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:35:54 2022] PUT /identity/v3/projects/bd593bc662964f8da5c990f1b7e04263/users/3a62c5de497f49ea90c339ec3b6c511b/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:35:54.311815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4d1e7-1df9-477b-9fb9-93f20826b608 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:54.312107 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4d1e7-1df9-477b-9fb9-93f20826b608 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:54.312385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4d1e7-1df9-477b-9fb9-93f20826b608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:54.339139 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-02c4d1e7-1df9-477b-9fb9-93f20826b608 None None] MFA Rules not processed for user `3a62c5de497f49ea90c339ec3b6c511b`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:35:54.340335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-02c4d1e7-1df9-477b-9fb9-93f20826b608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:54.408809 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1417/2834] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:35:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:35:54.420511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4ff89-bb54-4e75-bf91-9243474fc4a2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:54.422286 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b1f4ff89-bb54-4e75-bf91-9243474fc4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:35:54.479520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4ff89-bb54-4e75-bf91-9243474fc4a2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:54.483132 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc905801-0136-4600-8658-2907560b5dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:54.483728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905801-0136-4600-8658-2907560b5dfd service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:54.484007 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905801-0136-4600-8658-2907560b5dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:54.484277 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905801-0136-4600-8658-2907560b5dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:54.537377 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:35:54.542252 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc905801-0136-4600-8658-2907560b5dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:54.543983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc905801-0136-4600-8658-2907560b5dfd service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:54.553070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1418/2835] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:56.051512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4d1e7-1df9-477b-9fb9-93f20826b608 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:56.061727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4d1e7-1df9-477b-9fb9-93f20826b608 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:56.065324 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a5243-ab81-4f4b-a715-23bc3ea476bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:56.065918 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a5243-ab81-4f4b-a715-23bc3ea476bc service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:56.066225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a5243-ab81-4f4b-a715-23bc3ea476bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:56.066503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a5243-ab81-4f4b-a715-23bc3ea476bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:56.122039 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a5243-ab81-4f4b-a715-23bc3ea476bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:56.123414 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a5243-ab81-4f4b-a715-23bc3ea476bc service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:56.132079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1418/2836] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:35:56.668050 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc905801-0136-4600-8658-2907560b5dfd service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:35:56.676890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc905801-0136-4600-8658-2907560b5dfd service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:35:56.680511 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7db8a-7f0d-4ea7-a461-069d9bb60444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:35:56.681103 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7db8a-7f0d-4ea7-a461-069d9bb60444 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:35:56.681381 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7db8a-7f0d-4ea7-a461-069d9bb60444 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:35:56.681660 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7db8a-7f0d-4ea7-a461-069d9bb60444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:35:56.692328 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7db8a-7f0d-4ea7-a461-069d9bb60444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:35:56.694384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7db8a-7f0d-4ea7-a461-069d9bb60444 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:35:56.714169 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1419/2837] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:35:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:36:02.741537 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a5243-ab81-4f4b-a715-23bc3ea476bc service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:02.785579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:02.804163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a5243-ab81-4f4b-a715-23bc3ea476bc service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:02.808668 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:02.809287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:02.809563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:02.809832 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] PATH_INFO: `/v3/users/e4bf3b3c35c14e15ba0702eee03a61df` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:02.811560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4bf3b3c35c14e15ba0702eee03a61df)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:02.813462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:02.860058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4bf3b3c35c14e15ba0702eee03a61df'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:02.860721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4bf3b3c35c14e15ba0702eee03a61df'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:02.868112 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4bf3b3c35c14e15ba0702eee03a61df'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:02.868761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4bf3b3c35c14e15ba0702eee03a61df'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:02.875808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4bf3b3c35c14e15ba0702eee03a61df'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:02.893628 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1419/2838] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:36:02 2022] DELETE /identity/v3/users/e4bf3b3c35c14e15ba0702eee03a61df => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:02.898137 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a885a4b-f9c3-4938-bdd0-29dcc5d73148 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:02.898656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a885a4b-f9c3-4938-bdd0-29dcc5d73148 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:02.898759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a885a4b-f9c3-4938-bdd0-29dcc5d73148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:02.918943 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8a885a4b-f9c3-4938-bdd0-29dcc5d73148 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:02.920775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a885a4b-f9c3-4938-bdd0-29dcc5d73148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:02.983145 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1420/2839] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:02.993103 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:02.994533 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:03.050852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f8030-0366-43ac-99b1-9d7c67f93fe3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:03.054483 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1281f-ccca-4b60-8598-06eae7eeae70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:03.055085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1281f-ccca-4b60-8598-06eae7eeae70 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:03.055365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1281f-ccca-4b60-8598-06eae7eeae70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:03.055639 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1281f-ccca-4b60-8598-06eae7eeae70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:03.064466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1281f-ccca-4b60-8598-06eae7eeae70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:03.065881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1281f-ccca-4b60-8598-06eae7eeae70 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:03.075109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1420/2840] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:36:03.335499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a885a4b-f9c3-4938-bdd0-29dcc5d73148 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:03.337039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a885a4b-f9c3-4938-bdd0-29dcc5d73148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:03.351825 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a885a4b-f9c3-4938-bdd0-29dcc5d73148 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:03.357370 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:03.358276 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:03.358635 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:03.359017 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] PATH_INFO: `/v3/projects/92622c92d2aa423eaf4d4baf4a932423` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:03.360266 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] RBAC: Authorizing `identity:delete_project(project_id=92622c92d2aa423eaf4d4baf4a932423)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:03.362512 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:03.409329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92622c92d2aa423eaf4d4baf4a932423', 'request_id': 'req-30480525-f582-4e21-b9eb-233efa15819c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.416600 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92622c92d2aa423eaf4d4baf4a932423', 'request_id': 'req-30480525-f582-4e21-b9eb-233efa15819c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.422634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92622c92d2aa423eaf4d4baf4a932423', 'request_id': 'req-30480525-f582-4e21-b9eb-233efa15819c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.433619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] The token cache is being invalidate because project 92622c92d2aa423eaf4d4baf4a932423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:36:03.433911 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.434570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.436045 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1421/2841] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:36:03 2022] DELETE /identity/v3/projects/92622c92d2aa423eaf4d4baf4a932423 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:03.440051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1281f-ccca-4b60-8598-06eae7eeae70 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:03.449611 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1281f-ccca-4b60-8598-06eae7eeae70 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:03.504229 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:03.504778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:03.505059 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:03.505334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] PATH_INFO: `/v3/users/83022be705a4458b8fed7ae5d5412056` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:03.506469 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83022be705a4458b8fed7ae5d5412056)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:03.508386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:03.512820 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:03.553672 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83022be705a4458b8fed7ae5d5412056'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.554382 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83022be705a4458b8fed7ae5d5412056'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.561831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83022be705a4458b8fed7ae5d5412056'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.562478 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83022be705a4458b8fed7ae5d5412056'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.569211 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83022be705a4458b8fed7ae5d5412056'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:03.586333 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1421/2842] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:36:03 2022] DELETE /identity/v3/users/83022be705a4458b8fed7ae5d5412056 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:03.875571 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:03.877186 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:03.891710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30480525-f582-4e21-b9eb-233efa15819c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:03.947499 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:03.948218 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:03.948684 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:03.949027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] PATH_INFO: `/v3/projects/2766dc8d1815456b9118e9bbbab9a6fe` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:03.950390 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2766dc8d1815456b9118e9bbbab9a6fe)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:03.953063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:03.959703 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:04.006027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2766dc8d1815456b9118e9bbbab9a6fe', 'request_id': 'req-56fb67fb-d027-442f-aa9b-b3da33caf4a6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.015459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2766dc8d1815456b9118e9bbbab9a6fe', 'request_id': 'req-56fb67fb-d027-442f-aa9b-b3da33caf4a6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.021927 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2766dc8d1815456b9118e9bbbab9a6fe', 'request_id': 'req-56fb67fb-d027-442f-aa9b-b3da33caf4a6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.033338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] The token cache is being invalidate because project 2766dc8d1815456b9118e9bbbab9a6fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:36:04.033629 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.034320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.035789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1422/2843] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:36:03 2022] DELETE /identity/v3/projects/2766dc8d1815456b9118e9bbbab9a6fe => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:04.039792 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:04.041225 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:04.054266 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f00aa2-e5cb-4faf-be46-213af743c116 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:04.111742 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:04.112363 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:04.112641 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:04.112912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] PATH_INFO: `/v3/users/3a62c5de497f49ea90c339ec3b6c511b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:04.114038 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a62c5de497f49ea90c339ec3b6c511b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:04.115939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:04.160161 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a62c5de497f49ea90c339ec3b6c511b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.160805 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a62c5de497f49ea90c339ec3b6c511b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.167935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a62c5de497f49ea90c339ec3b6c511b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.168577 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a62c5de497f49ea90c339ec3b6c511b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.176105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a62c5de497f49ea90c339ec3b6c511b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.193406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1422/2844] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:36:04 2022] DELETE /identity/v3/users/3a62c5de497f49ea90c339ec3b6c511b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:04.463239 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:04.464664 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:04.480217 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb67fb-d027-442f-aa9b-b3da33caf4a6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:04.536307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:04.536991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:04.537218 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:04.537495 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] PATH_INFO: `/v3/projects/bd593bc662964f8da5c990f1b7e04263` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:04.538581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd593bc662964f8da5c990f1b7e04263)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:04.540493 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:04.591393 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd593bc662964f8da5c990f1b7e04263', 'request_id': 'req-4374c1bf-c123-411d-87e9-e602a6554724'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.598974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd593bc662964f8da5c990f1b7e04263', 'request_id': 'req-4374c1bf-c123-411d-87e9-e602a6554724'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.604547 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd593bc662964f8da5c990f1b7e04263', 'request_id': 'req-4374c1bf-c123-411d-87e9-e602a6554724'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.618398 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] The token cache is being invalidate because project bd593bc662964f8da5c990f1b7e04263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:36:04.618829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.619643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4374c1bf-c123-411d-87e9-e602a6554724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:04.621596 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1423/2845] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:36:04 2022] DELETE /identity/v3/projects/bd593bc662964f8da5c990f1b7e04263 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:05.004565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:05.006073 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:05.014674 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:05.020887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff93b4-5a25-4884-baa2-0f86be03c581 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:05.080168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:05.080825 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:05.081064 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:05.081338 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] PATH_INFO: `/v3/users/c260bd45a2474d1889b5d5da5682574f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:05.082489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c260bd45a2474d1889b5d5da5682574f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:05.084410 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:05.128348 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c260bd45a2474d1889b5d5da5682574f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.128982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c260bd45a2474d1889b5d5da5682574f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.136077 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c260bd45a2474d1889b5d5da5682574f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.136606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c260bd45a2474d1889b5d5da5682574f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.144048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c260bd45a2474d1889b5d5da5682574f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.160589 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1423/2846] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:36:05 2022] DELETE /identity/v3/users/c260bd45a2474d1889b5d5da5682574f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:05.165043 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91b9d0-b7dc-48a4-a1d7-f50c0231e434 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:05.165525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91b9d0-b7dc-48a4-a1d7-f50c0231e434 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:05.165624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91b9d0-b7dc-48a4-a1d7-f50c0231e434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:05.184546 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1e91b9d0-b7dc-48a4-a1d7-f50c0231e434 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:05.186297 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1e91b9d0-b7dc-48a4-a1d7-f50c0231e434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:05.204644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:05.244865 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1424/2847] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:05.254986 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:05.256409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:05.312388 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee9ff6-2ee1-4f77-a6ff-3b9a72d5d615 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:05.315945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c3558-92ad-4f37-b7ec-a8acdfcbd036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:05.316539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c3558-92ad-4f37-b7ec-a8acdfcbd036 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:05.316824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c3558-92ad-4f37-b7ec-a8acdfcbd036 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:05.317081 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c3558-92ad-4f37-b7ec-a8acdfcbd036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:05.325651 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c3558-92ad-4f37-b7ec-a8acdfcbd036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:05.327757 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c3558-92ad-4f37-b7ec-a8acdfcbd036 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:05.336812 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1424/2848] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:36:05.607106 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91b9d0-b7dc-48a4-a1d7-f50c0231e434 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:05.609243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1e91b9d0-b7dc-48a4-a1d7-f50c0231e434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:05.621821 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91b9d0-b7dc-48a4-a1d7-f50c0231e434 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:05.627956 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:05.628623 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:05.629032 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:05.629188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] PATH_INFO: `/v3/projects/edecb1d4976d4c8994c9a1b5b4d0ca84` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:05.630598 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edecb1d4976d4c8994c9a1b5b4d0ca84)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:05.632262 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:05.678156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edecb1d4976d4c8994c9a1b5b4d0ca84', 'request_id': 'req-6b0bc5ed-3d33-476c-af54-4ea695349a66'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.684904 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edecb1d4976d4c8994c9a1b5b4d0ca84', 'request_id': 'req-6b0bc5ed-3d33-476c-af54-4ea695349a66'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.691156 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edecb1d4976d4c8994c9a1b5b4d0ca84', 'request_id': 'req-6b0bc5ed-3d33-476c-af54-4ea695349a66'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.701978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] The token cache is being invalidate because project edecb1d4976d4c8994c9a1b5b4d0ca84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:36:05.702267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.702884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-6b0bc5ed-3d33-476c-af54-4ea695349a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:05.704322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1425/2849] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:36:05 2022] DELETE /identity/v3/projects/edecb1d4976d4c8994c9a1b5b4d0ca84 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:36:05.709317 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26bef-91bd-4aec-beb6-acbf04424656 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:05.709601 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26bef-91bd-4aec-beb6-acbf04424656 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:05.709874 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26bef-91bd-4aec-beb6-acbf04424656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:05.728499 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5fe26bef-91bd-4aec-beb6-acbf04424656 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:05.729678 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5fe26bef-91bd-4aec-beb6-acbf04424656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:05.748052 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:05.788973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1425/2850] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:05.798165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc23c08-fcd5-4635-b46c-844fb4e7bb22 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:05.798461 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc23c08-fcd5-4635-b46c-844fb4e7bb22 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:05.798746 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc23c08-fcd5-4635-b46c-844fb4e7bb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:05.818843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ebc23c08-fcd5-4635-b46c-844fb4e7bb22 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:05.820052 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ebc23c08-fcd5-4635-b46c-844fb4e7bb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:05.827931 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1426/2851] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:05.831972 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe26bef-91bd-4aec-beb6-acbf04424656 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:05.841296 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe26bef-91bd-4aec-beb6-acbf04424656 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:05.846267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dea3c-106f-49ed-8378-bcde49d2c33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:05.846847 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dea3c-106f-49ed-8378-bcde49d2c33a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:05.847125 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dea3c-106f-49ed-8378-bcde49d2c33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:05.847391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dea3c-106f-49ed-8378-bcde49d2c33a admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:05.848016 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dea3c-106f-49ed-8378-bcde49d2c33a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:36:05.848311 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dea3c-106f-49ed-8378-bcde49d2c33a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:05.849875 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dea3c-106f-49ed-8378-bcde49d2c33a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:05.862056 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1426/2852] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:36:05.866090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc23c08-fcd5-4635-b46c-844fb4e7bb22 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:05.875205 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc23c08-fcd5-4635-b46c-844fb4e7bb22 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:05.879830 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0219aa0-ae62-4d50-a44a-e7436f14a971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:05.880455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0219aa0-ae62-4d50-a44a-e7436f14a971 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:05.880737 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0219aa0-ae62-4d50-a44a-e7436f14a971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:05.881027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0219aa0-ae62-4d50-a44a-e7436f14a971 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:05.881631 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0219aa0-ae62-4d50-a44a-e7436f14a971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:05.883599 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0219aa0-ae62-4d50-a44a-e7436f14a971 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:05.910347 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1427/2853] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:36:05 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:36:05.914560 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dea3c-106f-49ed-8378-bcde49d2c33a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:05.925047 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dea3c-106f-49ed-8378-bcde49d2c33a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:05.978962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b38b8-6f60-4920-a4af-448d3712cb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:05.979531 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b38b8-6f60-4920-a4af-448d3712cb02 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:05.979775 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b38b8-6f60-4920-a4af-448d3712cb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:05.980048 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b38b8-6f60-4920-a4af-448d3712cb02 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:05.980670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b38b8-6f60-4920-a4af-448d3712cb02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:05.983245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b38b8-6f60-4920-a4af-448d3712cb02 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.003438 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1427/2854] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:36:05 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:36:06.007722 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0219aa0-ae62-4d50-a44a-e7436f14a971 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.017915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0219aa0-ae62-4d50-a44a-e7436f14a971 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.022574 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df276cd7-fe5b-4a55-b899-9534fbf20b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.023195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df276cd7-fe5b-4a55-b899-9534fbf20b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.023480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df276cd7-fe5b-4a55-b899-9534fbf20b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.023751 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df276cd7-fe5b-4a55-b899-9534fbf20b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.024388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df276cd7-fe5b-4a55-b899-9534fbf20b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.026002 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df276cd7-fe5b-4a55-b899-9534fbf20b0d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.036245 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1428/2855] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:36:06.039946 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b38b8-6f60-4920-a4af-448d3712cb02 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.050451 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b38b8-6f60-4920-a4af-448d3712cb02 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.055074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83496-d3cb-41a9-900c-c7e57e4c1cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.055663 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83496-d3cb-41a9-900c-c7e57e4c1cec admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.055944 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83496-d3cb-41a9-900c-c7e57e4c1cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.056219 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83496-d3cb-41a9-900c-c7e57e4c1cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.056851 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83496-d3cb-41a9-900c-c7e57e4c1cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.058470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83496-d3cb-41a9-900c-c7e57e4c1cec admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.069159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1428/2856] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:36:06.073084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df276cd7-fe5b-4a55-b899-9534fbf20b0d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.083469 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df276cd7-fe5b-4a55-b899-9534fbf20b0d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.087972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eccea-f8ef-4f59-ae86-607e81d8f963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.088606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eccea-f8ef-4f59-ae86-607e81d8f963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.088885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eccea-f8ef-4f59-ae86-607e81d8f963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.089155 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eccea-f8ef-4f59-ae86-607e81d8f963 admin admin] PATH_INFO: `/v3/projects/366479b0b9f64ea38928a218f898e2e0/users/13d5342dbfd44f2582e9f3588822a830/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.104528 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2eccea-f8ef-4f59-ae86-607e81d8f963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=366479b0b9f64ea38928a218f898e2e0, user_id=13d5342dbfd44f2582e9f3588822a830, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.107174 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2eccea-f8ef-4f59-ae86-607e81d8f963 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.120112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1429/2857] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:36:06 2022] PUT /identity/v3/projects/366479b0b9f64ea38928a218f898e2e0/users/13d5342dbfd44f2582e9f3588822a830/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:06.130244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05cb0a-7ef1-4670-9a7a-09956b4af6e3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.130710 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05cb0a-7ef1-4670-9a7a-09956b4af6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.130808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05cb0a-7ef1-4670-9a7a-09956b4af6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.157619 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2a05cb0a-7ef1-4670-9a7a-09956b4af6e3 None None] MFA Rules not processed for user `13d5342dbfd44f2582e9f3588822a830`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:06.158856 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2a05cb0a-7ef1-4670-9a7a-09956b4af6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:06.222208 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1429/2858] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:06.226577 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eccea-f8ef-4f59-ae86-607e81d8f963 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.270415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:06.288232 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eccea-f8ef-4f59-ae86-607e81d8f963 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.292735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8b94d-5a07-4dd8-b534-592957fb9db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.293325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8b94d-5a07-4dd8-b534-592957fb9db5 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.293601 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8b94d-5a07-4dd8-b534-592957fb9db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.293870 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8b94d-5a07-4dd8-b534-592957fb9db5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.294520 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8b94d-5a07-4dd8-b534-592957fb9db5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.296431 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8b94d-5a07-4dd8-b534-592957fb9db5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.321065 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1430/2859] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:36:06 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:36:06.325061 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05cb0a-7ef1-4670-9a7a-09956b4af6e3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.336155 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05cb0a-7ef1-4670-9a7a-09956b4af6e3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.352662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:06.394406 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9087-e01a-4d7d-9a01-4c28379f0bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.394992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9087-e01a-4d7d-9a01-4c28379f0bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.395271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9087-e01a-4d7d-9a01-4c28379f0bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.395540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9087-e01a-4d7d-9a01-4c28379f0bdf admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.396168 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e9087-e01a-4d7d-9a01-4c28379f0bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.398105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e9087-e01a-4d7d-9a01-4c28379f0bdf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.422491 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1430/2860] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:36:06 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:36:06.426489 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8b94d-5a07-4dd8-b534-592957fb9db5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.436789 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8b94d-5a07-4dd8-b534-592957fb9db5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.441320 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2655e87-9627-4282-bb78-63b622af5b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.441906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655e87-9627-4282-bb78-63b622af5b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.442211 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655e87-9627-4282-bb78-63b622af5b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.442480 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655e87-9627-4282-bb78-63b622af5b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.443336 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2655e87-9627-4282-bb78-63b622af5b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.444910 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2655e87-9627-4282-bb78-63b622af5b39 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.455648 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1431/2861] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:36:06.459606 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9087-e01a-4d7d-9a01-4c28379f0bdf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.470079 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9087-e01a-4d7d-9a01-4c28379f0bdf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.474666 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7ae5e-ba43-4c85-858c-62a3e2bf8ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.475255 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7ae5e-ba43-4c85-858c-62a3e2bf8ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.475536 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7ae5e-ba43-4c85-858c-62a3e2bf8ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.475808 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7ae5e-ba43-4c85-858c-62a3e2bf8ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.476439 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7ae5e-ba43-4c85-858c-62a3e2bf8ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.478049 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7ae5e-ba43-4c85-858c-62a3e2bf8ed6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.488123 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1431/2862] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:36:06.492109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2655e87-9627-4282-bb78-63b622af5b39 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.501803 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2655e87-9627-4282-bb78-63b622af5b39 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.506325 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3726ca8-fae9-4308-9c55-000baced76bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.506961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3726ca8-fae9-4308-9c55-000baced76bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.507235 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3726ca8-fae9-4308-9c55-000baced76bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.507505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3726ca8-fae9-4308-9c55-000baced76bd admin admin] PATH_INFO: `/v3/projects/cedc04adb68f4be6b0f875187a491de5/users/4c786ac6d8cd42ddadb68cbd4ff56a4e/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.522608 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3726ca8-fae9-4308-9c55-000baced76bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cedc04adb68f4be6b0f875187a491de5, user_id=4c786ac6d8cd42ddadb68cbd4ff56a4e, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.525194 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3726ca8-fae9-4308-9c55-000baced76bd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.538360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1432/2863] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:36:06 2022] PUT /identity/v3/projects/cedc04adb68f4be6b0f875187a491de5/users/4c786ac6d8cd42ddadb68cbd4ff56a4e/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 05:36:06.549448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69756c-337b-490b-a947-9ad164ef176e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.549734 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69756c-337b-490b-a947-9ad164ef176e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.550075 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69756c-337b-490b-a947-9ad164ef176e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.579373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-dc69756c-337b-490b-a947-9ad164ef176e None None] MFA Rules not processed for user `4c786ac6d8cd42ddadb68cbd4ff56a4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:06.580571 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dc69756c-337b-490b-a947-9ad164ef176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:06.646040 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1432/2864] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:06.650436 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3726ca8-fae9-4308-9c55-000baced76bd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.710170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3726ca8-fae9-4308-9c55-000baced76bd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.714706 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71292a4f-6974-47dc-8184-fcf18a48b70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.715295 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71292a4f-6974-47dc-8184-fcf18a48b70a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.715578 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71292a4f-6974-47dc-8184-fcf18a48b70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.715848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71292a4f-6974-47dc-8184-fcf18a48b70a admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.716460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71292a4f-6974-47dc-8184-fcf18a48b70a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.718408 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71292a4f-6974-47dc-8184-fcf18a48b70a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.744315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1433/2865] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:36:06 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:36:06.749101 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69756c-337b-490b-a947-9ad164ef176e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.759566 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69756c-337b-490b-a947-9ad164ef176e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.814688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2789c-9afd-4bdb-ab7f-3913892f6287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.815244 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2789c-9afd-4bdb-ab7f-3913892f6287 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.815527 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2789c-9afd-4bdb-ab7f-3913892f6287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.815802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2789c-9afd-4bdb-ab7f-3913892f6287 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.816430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2789c-9afd-4bdb-ab7f-3913892f6287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.818371 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2789c-9afd-4bdb-ab7f-3913892f6287 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.839085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1433/2866] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:36:06 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:36:06.843056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71292a4f-6974-47dc-8184-fcf18a48b70a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.853139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71292a4f-6974-47dc-8184-fcf18a48b70a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.857731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b35674-bf33-491b-9079-aa934f62db91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.858365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b35674-bf33-491b-9079-aa934f62db91 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.858634 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b35674-bf33-491b-9079-aa934f62db91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.858906 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b35674-bf33-491b-9079-aa934f62db91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.859533 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b35674-bf33-491b-9079-aa934f62db91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.861372 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b35674-bf33-491b-9079-aa934f62db91 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.871748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1434/2867] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:36:06.876179 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2789c-9afd-4bdb-ab7f-3913892f6287 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.885955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2789c-9afd-4bdb-ab7f-3913892f6287 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.891268 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face0098-dee9-4976-b0ed-fd653d123581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.891913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face0098-dee9-4976-b0ed-fd653d123581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.892196 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face0098-dee9-4976-b0ed-fd653d123581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.892474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face0098-dee9-4976-b0ed-fd653d123581 admin admin] PATH_INFO: `/v3/projects/bc7d670448554c1c93e72b60601ee911/users/bc33a588336946acafd41900fc6dbe95/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.906815 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face0098-dee9-4976-b0ed-fd653d123581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc7d670448554c1c93e72b60601ee911, user_id=bc33a588336946acafd41900fc6dbe95, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.909448 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face0098-dee9-4976-b0ed-fd653d123581 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:06.911635 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:06.922563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1434/2868] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:36:06 2022] PUT /identity/v3/projects/bc7d670448554c1c93e72b60601ee911/users/bc33a588336946acafd41900fc6dbe95/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:06.926487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b35674-bf33-491b-9079-aa934f62db91 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:06.936806 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b35674-bf33-491b-9079-aa934f62db91 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:06.992856 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86118fc-4d5d-4f62-984e-408dd390f6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:06.993473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86118fc-4d5d-4f62-984e-408dd390f6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:06.993755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86118fc-4d5d-4f62-984e-408dd390f6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:06.994081 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86118fc-4d5d-4f62-984e-408dd390f6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:06.994712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86118fc-4d5d-4f62-984e-408dd390f6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:06.996285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86118fc-4d5d-4f62-984e-408dd390f6d2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:07.006307 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1435/2869] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:36:07.010397 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face0098-dee9-4976-b0ed-fd653d123581 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:07.020335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face0098-dee9-4976-b0ed-fd653d123581 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:07.024877 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8e398-7415-4036-a933-d4331cb9d82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:07.025510 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8e398-7415-4036-a933-d4331cb9d82b admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:07.025785 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8e398-7415-4036-a933-d4331cb9d82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:07.026094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8e398-7415-4036-a933-d4331cb9d82b admin admin] PATH_INFO: `/v3/projects/bc7d670448554c1c93e72b60601ee911/users/bc33a588336946acafd41900fc6dbe95/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:07.027954 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8e398-7415-4036-a933-d4331cb9d82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc7d670448554c1c93e72b60601ee911, user_id=bc33a588336946acafd41900fc6dbe95, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:07.030590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8e398-7415-4036-a933-d4331cb9d82b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:07.040427 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1435/2870] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:36:07 2022] PUT /identity/v3/projects/bc7d670448554c1c93e72b60601ee911/users/bc33a588336946acafd41900fc6dbe95/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:07.103750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25f282-aeb1-498c-8de6-0c5ef31d5831 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:07.104039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25f282-aeb1-498c-8de6-0c5ef31d5831 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:07.104346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25f282-aeb1-498c-8de6-0c5ef31d5831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:07.113195 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:07.134147 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3c25f282-aeb1-498c-8de6-0c5ef31d5831 None None] MFA Rules not processed for user `bc33a588336946acafd41900fc6dbe95`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:07.135361 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3c25f282-aeb1-498c-8de6-0c5ef31d5831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:07.208953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1436/2871] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:07.220114 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8e398-7415-4036-a933-d4331cb9d82b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:07.221797 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-02e8e398-7415-4036-a933-d4331cb9d82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:07.277768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8e398-7415-4036-a933-d4331cb9d82b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:07.281516 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a58b5-ccfb-4e83-b222-58b306b0e075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:07.282166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a58b5-ccfb-4e83-b222-58b306b0e075 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:07.282450 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a58b5-ccfb-4e83-b222-58b306b0e075 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:07.282722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a58b5-ccfb-4e83-b222-58b306b0e075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:07.337722 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a58b5-ccfb-4e83-b222-58b306b0e075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:07.339221 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a58b5-ccfb-4e83-b222-58b306b0e075 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:07.348386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1436/2872] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:36:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:36:08.848015 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25f282-aeb1-498c-8de6-0c5ef31d5831 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:08.849329 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3c25f282-aeb1-498c-8de6-0c5ef31d5831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:08.862161 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25f282-aeb1-498c-8de6-0c5ef31d5831 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:08.865661 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00def6f5-b2c5-4d64-921b-2aada2396cf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:08.866289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00def6f5-b2c5-4d64-921b-2aada2396cf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:08.866568 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00def6f5-b2c5-4d64-921b-2aada2396cf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:08.866838 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00def6f5-b2c5-4d64-921b-2aada2396cf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:08.921755 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00def6f5-b2c5-4d64-921b-2aada2396cf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:08.923202 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00def6f5-b2c5-4d64-921b-2aada2396cf8 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:08.931832 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1437/2873] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:36:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:36:09.465315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a58b5-ccfb-4e83-b222-58b306b0e075 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:09.474708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a58b5-ccfb-4e83-b222-58b306b0e075 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:09.478362 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7dce4-20cb-4d7c-b539-e94dad243309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:09.478956 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7dce4-20cb-4d7c-b539-e94dad243309 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:09.479235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7dce4-20cb-4d7c-b539-e94dad243309 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:09.479506 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7dce4-20cb-4d7c-b539-e94dad243309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:09.488582 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7dce4-20cb-4d7c-b539-e94dad243309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:09.490011 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7dce4-20cb-4d7c-b539-e94dad243309 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:09.499179 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1437/2874] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:36:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:36:13.363299 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00def6f5-b2c5-4d64-921b-2aada2396cf8 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:13.426622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00def6f5-b2c5-4d64-921b-2aada2396cf8 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:13.431090 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:13.431716 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:13.431996 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:13.432273 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] PATH_INFO: `/v3/users/13d5342dbfd44f2582e9f3588822a830` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:13.433475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d5342dbfd44f2582e9f3588822a830)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:13.435983 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:13.468588 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:13.482839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d5342dbfd44f2582e9f3588822a830'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:13.489734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d5342dbfd44f2582e9f3588822a830'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:13.490407 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d5342dbfd44f2582e9f3588822a830'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:13.491026 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d5342dbfd44f2582e9f3588822a830'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:13.498491 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d5342dbfd44f2582e9f3588822a830'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:13.515930 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1438/2875] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:36:13 2022] DELETE /identity/v3/users/13d5342dbfd44f2582e9f3588822a830 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:13.520547 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d76f73-b652-4d2f-a4a8-b647a6278b26 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:13.521031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d76f73-b652-4d2f-a4a8-b647a6278b26 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:13.521140 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d76f73-b652-4d2f-a4a8-b647a6278b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:13.539755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-21d76f73-b652-4d2f-a4a8-b647a6278b26 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:13.540955 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-21d76f73-b652-4d2f-a4a8-b647a6278b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:13.599517 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1438/2876] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:13.609330 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:13.610774 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:13.670921 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaf434-683f-4bf6-a0d4-515cb0824f6b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:13.674538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fbd92-a03e-42f5-9c37-f8dc6ca57f29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:13.675135 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fbd92-a03e-42f5-9c37-f8dc6ca57f29 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:13.675412 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fbd92-a03e-42f5-9c37-f8dc6ca57f29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:13.675693 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fbd92-a03e-42f5-9c37-f8dc6ca57f29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:13.684254 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fbd92-a03e-42f5-9c37-f8dc6ca57f29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:13.686272 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fbd92-a03e-42f5-9c37-f8dc6ca57f29 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:13.695356 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1439/2877] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:36:13.960898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d76f73-b652-4d2f-a4a8-b647a6278b26 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:13.962458 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-21d76f73-b652-4d2f-a4a8-b647a6278b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:13.977278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d76f73-b652-4d2f-a4a8-b647a6278b26 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:13.983746 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:13.984658 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:13.984937 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:13.985212 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] PATH_INFO: `/v3/projects/366479b0b9f64ea38928a218f898e2e0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:13.986709 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=366479b0b9f64ea38928a218f898e2e0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:13.988649 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:14.005670 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:14.036310 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '366479b0b9f64ea38928a218f898e2e0', 'request_id': 'req-c158e4f0-438a-4e79-bdcf-908ad09b7f06'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.041912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '366479b0b9f64ea38928a218f898e2e0', 'request_id': 'req-c158e4f0-438a-4e79-bdcf-908ad09b7f06'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.049305 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '366479b0b9f64ea38928a218f898e2e0', 'request_id': 'req-c158e4f0-438a-4e79-bdcf-908ad09b7f06'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.060768 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] The token cache is being invalidate because project 366479b0b9f64ea38928a218f898e2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:36:14.061063 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.061721 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.063263 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1439/2878] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:36:13 2022] DELETE /identity/v3/projects/366479b0b9f64ea38928a218f898e2e0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:14.067283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fbd92-a03e-42f5-9c37-f8dc6ca57f29 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:14.077351 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fbd92-a03e-42f5-9c37-f8dc6ca57f29 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:14.132127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:14.132749 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:14.133031 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:14.133305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] PATH_INFO: `/v3/users/4c786ac6d8cd42ddadb68cbd4ff56a4e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:14.134439 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c786ac6d8cd42ddadb68cbd4ff56a4e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:14.136338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:14.179798 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c786ac6d8cd42ddadb68cbd4ff56a4e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.183630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:14.187453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c786ac6d8cd42ddadb68cbd4ff56a4e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.188240 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c786ac6d8cd42ddadb68cbd4ff56a4e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.189311 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c786ac6d8cd42ddadb68cbd4ff56a4e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.197072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c786ac6d8cd42ddadb68cbd4ff56a4e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.214338 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1440/2879] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:36:14 2022] DELETE /identity/v3/users/4c786ac6d8cd42ddadb68cbd4ff56a4e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:14.473714 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:14.475365 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:14.489526 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158e4f0-438a-4e79-bdcf-908ad09b7f06 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:14.544344 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:14.545546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:14.545804 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:14.546105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] PATH_INFO: `/v3/projects/cedc04adb68f4be6b0f875187a491de5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:14.547257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cedc04adb68f4be6b0f875187a491de5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:14.549142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:14.595037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cedc04adb68f4be6b0f875187a491de5', 'request_id': 'req-fdce0033-9bb6-4d06-a708-36f221032dd5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.600659 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cedc04adb68f4be6b0f875187a491de5', 'request_id': 'req-fdce0033-9bb6-4d06-a708-36f221032dd5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.608643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cedc04adb68f4be6b0f875187a491de5', 'request_id': 'req-fdce0033-9bb6-4d06-a708-36f221032dd5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.620044 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] The token cache is being invalidate because project cedc04adb68f4be6b0f875187a491de5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:36:14.620328 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.620941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.622564 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1440/2880] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:36:14 2022] DELETE /identity/v3/projects/cedc04adb68f4be6b0f875187a491de5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:14.627093 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:14.628522 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:14.641251 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9f045-0e49-440b-9b88-730c4826ece2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:14.696366 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:14.696923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:14.697224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:14.697498 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] PATH_INFO: `/v3/users/bc33a588336946acafd41900fc6dbe95` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:14.698657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc33a588336946acafd41900fc6dbe95)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:14.700559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:14.744677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc33a588336946acafd41900fc6dbe95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.752699 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc33a588336946acafd41900fc6dbe95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.753473 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc33a588336946acafd41900fc6dbe95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.754105 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc33a588336946acafd41900fc6dbe95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.760991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-499abf2f-aa6d-43ab-ad27-54945f3d0815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc33a588336946acafd41900fc6dbe95'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:14.777742 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1441/2881] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:36:14 2022] DELETE /identity/v3/users/bc33a588336946acafd41900fc6dbe95 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:15.043291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:15.046754 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:15.063426 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce0033-9bb6-4d06-a708-36f221032dd5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:15.096912 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:15.122172 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:15.122857 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.123138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.123412 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] PATH_INFO: `/v3/projects/bc7d670448554c1c93e72b60601ee911` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.124518 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc7d670448554c1c93e72b60601ee911)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:15.126610 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:15.170546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc7d670448554c1c93e72b60601ee911', 'request_id': 'req-f95837b2-677f-4830-a2ee-cc687ca6151c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:15.175873 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc7d670448554c1c93e72b60601ee911', 'request_id': 'req-f95837b2-677f-4830-a2ee-cc687ca6151c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:15.183462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc7d670448554c1c93e72b60601ee911', 'request_id': 'req-f95837b2-677f-4830-a2ee-cc687ca6151c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:15.194070 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] The token cache is being invalidate because project bc7d670448554c1c93e72b60601ee911 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:36:15.194359 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:15.194987 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f95837b2-677f-4830-a2ee-cc687ca6151c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:36:15.196420 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1441/2882] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:36:15 2022] DELETE /identity/v3/projects/bc7d670448554c1c93e72b60601ee911 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:15.202104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45518cf-1b95-4fed-b9f7-a9dc2ac39dde None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.202396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45518cf-1b95-4fed-b9f7-a9dc2ac39dde None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.202700 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45518cf-1b95-4fed-b9f7-a9dc2ac39dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.222609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e45518cf-1b95-4fed-b9f7-a9dc2ac39dde None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:15.223752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e45518cf-1b95-4fed-b9f7-a9dc2ac39dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:15.266477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:15.283282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1442/2883] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:15.291093 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd468cd-e8dd-4858-bf40-dd6205083d4c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.291407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd468cd-e8dd-4858-bf40-dd6205083d4c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.291681 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd468cd-e8dd-4858-bf40-dd6205083d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.310833 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fcd468cd-e8dd-4858-bf40-dd6205083d4c None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:15.312020 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fcd468cd-e8dd-4858-bf40-dd6205083d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:15.319675 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1442/2884] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:15.323913 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45518cf-1b95-4fed-b9f7-a9dc2ac39dde None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:15.333985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45518cf-1b95-4fed-b9f7-a9dc2ac39dde None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:15.338570 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acbae1-cb97-4134-80a3-e78b71e6c658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:15.339167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acbae1-cb97-4134-80a3-e78b71e6c658 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.339450 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acbae1-cb97-4134-80a3-e78b71e6c658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.339729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acbae1-cb97-4134-80a3-e78b71e6c658 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.340424 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acbae1-cb97-4134-80a3-e78b71e6c658 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 05:36:15.340732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acbae1-cb97-4134-80a3-e78b71e6c658 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:15.342976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acbae1-cb97-4134-80a3-e78b71e6c658 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:15.354365 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1443/2885] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Sep 21 05:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 05:36:15.359337 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd468cd-e8dd-4858-bf40-dd6205083d4c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:15.368585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd468cd-e8dd-4858-bf40-dd6205083d4c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:15.373727 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175cdf1-1d64-4a83-8a2a-83caa1196ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:15.374340 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175cdf1-1d64-4a83-8a2a-83caa1196ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.374617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175cdf1-1d64-4a83-8a2a-83caa1196ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.374888 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175cdf1-1d64-4a83-8a2a-83caa1196ad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.375503 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175cdf1-1d64-4a83-8a2a-83caa1196ad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:15.377399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175cdf1-1d64-4a83-8a2a-83caa1196ad4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:15.403167 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1443/2886] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:36:15 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:36:15.407289 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acbae1-cb97-4134-80a3-e78b71e6c658 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:15.417094 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acbae1-cb97-4134-80a3-e78b71e6c658 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:15.471993 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea15ad-ecf3-413f-86c3-b58b2baae909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:15.472589 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea15ad-ecf3-413f-86c3-b58b2baae909 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.472923 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea15ad-ecf3-413f-86c3-b58b2baae909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.473142 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea15ad-ecf3-413f-86c3-b58b2baae909 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.473769 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea15ad-ecf3-413f-86c3-b58b2baae909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:15.475677 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea15ad-ecf3-413f-86c3-b58b2baae909 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:15.497091 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1444/2887] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:36:15 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:36:15.501781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175cdf1-1d64-4a83-8a2a-83caa1196ad4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:15.512193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175cdf1-1d64-4a83-8a2a-83caa1196ad4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:15.517302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcd704-f79a-4def-8391-b87767e801ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:15.517953 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcd704-f79a-4def-8391-b87767e801ff admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.518239 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcd704-f79a-4def-8391-b87767e801ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.518510 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcd704-f79a-4def-8391-b87767e801ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.519148 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcd704-f79a-4def-8391-b87767e801ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:15.520736 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcd704-f79a-4def-8391-b87767e801ff admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:15.530573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1444/2888] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:36:15.534396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea15ad-ecf3-413f-86c3-b58b2baae909 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:15.544059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea15ad-ecf3-413f-86c3-b58b2baae909 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:15.549114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86558fe-22c5-45c8-b212-2d1bc03ee3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:15.549689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86558fe-22c5-45c8-b212-2d1bc03ee3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.549991 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86558fe-22c5-45c8-b212-2d1bc03ee3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.550267 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86558fe-22c5-45c8-b212-2d1bc03ee3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.550891 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86558fe-22c5-45c8-b212-2d1bc03ee3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:15.552453 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86558fe-22c5-45c8-b212-2d1bc03ee3e0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:15.564459 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1445/2889] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:36:15.568561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcd704-f79a-4def-8391-b87767e801ff admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:15.578515 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcd704-f79a-4def-8391-b87767e801ff admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:15.583136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d3052-766e-43ee-a342-fbc33306c2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:15.583782 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d3052-766e-43ee-a342-fbc33306c2a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.584060 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d3052-766e-43ee-a342-fbc33306c2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.584335 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d3052-766e-43ee-a342-fbc33306c2a0 admin admin] PATH_INFO: `/v3/projects/9dc6882f51ef41cd9d276e79a85bd868/users/0c3f1c1971df49e195a998d55834ff48/roles/d36487377a664531a7f2e06f2f648a6f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.599563 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d3052-766e-43ee-a342-fbc33306c2a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc6882f51ef41cd9d276e79a85bd868, user_id=0c3f1c1971df49e195a998d55834ff48, role_id=d36487377a664531a7f2e06f2f648a6f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:15.602264 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d3052-766e-43ee-a342-fbc33306c2a0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:15.615460 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1445/2890] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:36:15 2022] PUT /identity/v3/projects/9dc6882f51ef41cd9d276e79a85bd868/users/0c3f1c1971df49e195a998d55834ff48/roles/d36487377a664531a7f2e06f2f648a6f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:15.620503 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079f490-4de7-48db-92bd-49ab90dfda20 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.620797 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079f490-4de7-48db-92bd-49ab90dfda20 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.621075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079f490-4de7-48db-92bd-49ab90dfda20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.639750 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5079f490-4de7-48db-92bd-49ab90dfda20 None None] MFA Rules not processed for user `493af8396fb3419ea552b2a7642f7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:15.640931 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5079f490-4de7-48db-92bd-49ab90dfda20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:15.700401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1446/2891] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:15.710654 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d3052-766e-43ee-a342-fbc33306c2a0 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:15.712405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7e5d3052-766e-43ee-a342-fbc33306c2a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:15.768191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d3052-766e-43ee-a342-fbc33306c2a0 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:15.771741 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c06758-6eb7-454f-87f5-e735ba5d1d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:15.772342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c06758-6eb7-454f-87f5-e735ba5d1d2e service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:15.772620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c06758-6eb7-454f-87f5-e735ba5d1d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:15.772898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c06758-6eb7-454f-87f5-e735ba5d1d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:15.783035 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c06758-6eb7-454f-87f5-e735ba5d1d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:15.784470 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c06758-6eb7-454f-87f5-e735ba5d1d2e service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:15.794191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1446/2892] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:36:19.499411 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab62ec-c813-4a27-a620-8d60f6a89c2f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:19.499980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab62ec-c813-4a27-a620-8d60f6a89c2f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:19.499980 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab62ec-c813-4a27-a620-8d60f6a89c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:19.528264 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-38ab62ec-c813-4a27-a620-8d60f6a89c2f None None] MFA Rules not processed for user `0c3f1c1971df49e195a998d55834ff48`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:19.530075 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-38ab62ec-c813-4a27-a620-8d60f6a89c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:19.584197 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:19.614538 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1447/2893] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:19.618752 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c06758-6eb7-454f-87f5-e735ba5d1d2e service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:19.628617 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:19.633287 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c06758-6eb7-454f-87f5-e735ba5d1d2e service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:19.642257 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe3d93-99b8-4527-a9b1-0f3a811a33d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:19.642897 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe3d93-99b8-4527-a9b1-0f3a811a33d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:19.643175 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe3d93-99b8-4527-a9b1-0f3a811a33d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:19.643444 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe3d93-99b8-4527-a9b1-0f3a811a33d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:19.644095 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe3d93-99b8-4527-a9b1-0f3a811a33d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:19.646085 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe3d93-99b8-4527-a9b1-0f3a811a33d1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:19.688989 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1447/2894] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Sep 21 05:36:19 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 05:36:19.693281 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab62ec-c813-4a27-a620-8d60f6a89c2f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:19.706230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab62ec-c813-4a27-a620-8d60f6a89c2f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:19.780263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af15f5a-eedf-433a-b160-f327fa23fa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:19.780962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af15f5a-eedf-433a-b160-f327fa23fa69 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:19.781308 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af15f5a-eedf-433a-b160-f327fa23fa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:19.781652 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af15f5a-eedf-433a-b160-f327fa23fa69 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:19.782423 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af15f5a-eedf-433a-b160-f327fa23fa69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:19.784388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af15f5a-eedf-433a-b160-f327fa23fa69 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:19.811139 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1448/2895] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Sep 21 05:36:19 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 05:36:19.815474 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe3d93-99b8-4527-a9b1-0f3a811a33d1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:19.826761 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe3d93-99b8-4527-a9b1-0f3a811a33d1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:19.831402 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869005b7-9fd7-47ed-bd4b-ca0e056d1ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:19.832011 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869005b7-9fd7-47ed-bd4b-ca0e056d1ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:19.832289 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869005b7-9fd7-47ed-bd4b-ca0e056d1ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:19.832561 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869005b7-9fd7-47ed-bd4b-ca0e056d1ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:19.833200 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869005b7-9fd7-47ed-bd4b-ca0e056d1ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:19.835138 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869005b7-9fd7-47ed-bd4b-ca0e056d1ea6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:19.847183 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1448/2896] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 05:36:19.851458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af15f5a-eedf-433a-b160-f327fa23fa69 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:19.864481 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af15f5a-eedf-433a-b160-f327fa23fa69 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:19.870592 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7ef31-7a89-4c1e-a3dc-ad0737e7a0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:19.871388 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7ef31-7a89-4c1e-a3dc-ad0737e7a0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:19.871734 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7ef31-7a89-4c1e-a3dc-ad0737e7a0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:19.872083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7ef31-7a89-4c1e-a3dc-ad0737e7a0b3 admin admin] PATH_INFO: `/v3/projects/2deeb2d3f51d450694d04532abadcdcb/users/e53ba401eaed45e1a6d614946f5c17b3/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:19.891005 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba7ef31-7a89-4c1e-a3dc-ad0737e7a0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2deeb2d3f51d450694d04532abadcdcb, user_id=e53ba401eaed45e1a6d614946f5c17b3, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:19.894839 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba7ef31-7a89-4c1e-a3dc-ad0737e7a0b3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:19.909046 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1449/2897] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:36:19 2022] PUT /identity/v3/projects/2deeb2d3f51d450694d04532abadcdcb/users/e53ba401eaed45e1a6d614946f5c17b3/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:19.913991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869005b7-9fd7-47ed-bd4b-ca0e056d1ea6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:19.929260 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869005b7-9fd7-47ed-bd4b-ca0e056d1ea6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:19.995756 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a32e1f-6b44-4281-a3cc-93f87ed21904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:19.996421 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a32e1f-6b44-4281-a3cc-93f87ed21904 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:19.996642 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a32e1f-6b44-4281-a3cc-93f87ed21904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:19.996958 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a32e1f-6b44-4281-a3cc-93f87ed21904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:19.997620 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a32e1f-6b44-4281-a3cc-93f87ed21904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:19.999763 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a32e1f-6b44-4281-a3cc-93f87ed21904 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:20.011119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1449/2898] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Sep 21 05:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 05:36:20.015367 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7ef31-7a89-4c1e-a3dc-ad0737e7a0b3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:20.030196 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7ef31-7a89-4c1e-a3dc-ad0737e7a0b3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:20.037035 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb374a8-246f-4c15-8d42-d5f33313198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:20.037814 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb374a8-246f-4c15-8d42-d5f33313198f admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:20.038218 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb374a8-246f-4c15-8d42-d5f33313198f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:20.039059 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb374a8-246f-4c15-8d42-d5f33313198f admin admin] PATH_INFO: `/v3/projects/2deeb2d3f51d450694d04532abadcdcb/users/e53ba401eaed45e1a6d614946f5c17b3/roles/51e166ed71fd44f8ab08c961b23a2118` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:20.046419 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb374a8-246f-4c15-8d42-d5f33313198f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2deeb2d3f51d450694d04532abadcdcb, user_id=e53ba401eaed45e1a6d614946f5c17b3, role_id=51e166ed71fd44f8ab08c961b23a2118)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:20.049083 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb374a8-246f-4c15-8d42-d5f33313198f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:20.065883 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1450/2899] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Sep 21 05:36:20 2022] PUT /identity/v3/projects/2deeb2d3f51d450694d04532abadcdcb/users/e53ba401eaed45e1a6d614946f5c17b3/roles/51e166ed71fd44f8ab08c961b23a2118 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:36:24.655220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138598e-b01d-41f3-b3fe-cb530a1ff3c1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:24.655764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138598e-b01d-41f3-b3fe-cb530a1ff3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:24.655764 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138598e-b01d-41f3-b3fe-cb530a1ff3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:24.688164 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3138598e-b01d-41f3-b3fe-cb530a1ff3c1 None None] MFA Rules not processed for user `e53ba401eaed45e1a6d614946f5c17b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:24.689778 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3138598e-b01d-41f3-b3fe-cb530a1ff3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:24.765504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1450/2900] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:24.786283 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb374a8-246f-4c15-8d42-d5f33313198f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:24.788781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9eb374a8-246f-4c15-8d42-d5f33313198f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:24.876164 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb374a8-246f-4c15-8d42-d5f33313198f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:24.881681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b751e-8976-43c1-8271-e028832298c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:24.882332 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b751e-8976-43c1-8271-e028832298c5 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:24.882619 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b751e-8976-43c1-8271-e028832298c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:24.882890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b751e-8976-43c1-8271-e028832298c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:24.918342 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:24.945141 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b751e-8976-43c1-8271-e028832298c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:24.946781 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b751e-8976-43c1-8271-e028832298c5 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:24.956773 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1451/2901] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:36:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:36:25.922907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4ed55-4e4c-451c-b6f3-2969f2a7a072 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:25.923430 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4ed55-4e4c-451c-b6f3-2969f2a7a072 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:25.923640 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4ed55-4e4c-451c-b6f3-2969f2a7a072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:25.945303 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-48d4ed55-4e4c-451c-b6f3-2969f2a7a072 None None] MFA Rules not processed for user `0c3f1c1971df49e195a998d55834ff48`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 05:36:25.946699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-48d4ed55-4e4c-451c-b6f3-2969f2a7a072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:25.953898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1451/2902] 10.222.0.18 () {58 vars in 958 bytes} [Wed Sep 21 05:36:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 05:36:34.952537 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=12 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:35.938802 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:36:52.765984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b751e-8976-43c1-8271-e028832298c5 service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:36:52.767375 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-171b751e-8976-43c1-8271-e028832298c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:52.822396 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b751e-8976-43c1-8271-e028832298c5 service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:36:52.825912 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd31204-ed3a-42bd-b611-fe784e00bd76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:36:52.826525 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd31204-ed3a-42bd-b611-fe784e00bd76 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:36:52.826801 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd31204-ed3a-42bd-b611-fe784e00bd76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:36:52.827068 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd31204-ed3a-42bd-b611-fe784e00bd76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:36:52.828704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bbd31204-ed3a-42bd-b611-fe784e00bd76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:36:52.885462 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd31204-ed3a-42bd-b611-fe784e00bd76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:36:52.886915 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd31204-ed3a-42bd-b611-fe784e00bd76 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:36:52.895880 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1452/2903] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:36:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:37:02.891704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=48 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:37:12.349920 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4ed55-4e4c-451c-b6f3-2969f2a7a072 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:37:12.403119 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4ed55-4e4c-451c-b6f3-2969f2a7a072 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:37:12.407144 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:37:12.407733 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:37:12.408008 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:37:12.408274 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:37:12.410007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:37:12.423142 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:37:12.424578 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:37:12.435725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1452/2904] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:37:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:37:13.620064 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd31204-ed3a-42bd-b611-fe784e00bd76 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:37:13.630497 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd31204-ed3a-42bd-b611-fe784e00bd76 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:37:13.634085 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352604e9-31fb-43b5-a51c-a06c265b2d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:37:13.634685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352604e9-31fb-43b5-a51c-a06c265b2d81 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:37:13.634985 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352604e9-31fb-43b5-a51c-a06c265b2d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:37:13.635257 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352604e9-31fb-43b5-a51c-a06c265b2d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:37:13.636992 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-352604e9-31fb-43b5-a51c-a06c265b2d81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:37:13.651656 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352604e9-31fb-43b5-a51c-a06c265b2d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:37:13.653104 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352604e9-31fb-43b5-a51c-a06c265b2d81 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:37:13.664864 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1453/2905] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:37:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:37:22.430758 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=26 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:37:23.659874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:37:44.943473 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:37:44.944992 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:37:44.955708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1c2be-4e67-42af-bb0a-a10cf9633c6a service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:37:44.959342 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef019ce4-70de-4ff5-90fd-cae015199272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:37:44.959935 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef019ce4-70de-4ff5-90fd-cae015199272 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:37:44.960222 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef019ce4-70de-4ff5-90fd-cae015199272 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:37:44.960497 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef019ce4-70de-4ff5-90fd-cae015199272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:37:44.962193 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ef019ce4-70de-4ff5-90fd-cae015199272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:37:44.971982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef019ce4-70de-4ff5-90fd-cae015199272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:37:44.973394 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef019ce4-70de-4ff5-90fd-cae015199272 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:37:44.981917 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1453/2906] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:37:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:37:47.924343 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352604e9-31fb-43b5-a51c-a06c265b2d81 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:37:47.933127 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352604e9-31fb-43b5-a51c-a06c265b2d81 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:37:47.936730 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee589e17-9895-4b13-b44f-91e76529f31b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:37:47.937324 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee589e17-9895-4b13-b44f-91e76529f31b service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:37:47.937606 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee589e17-9895-4b13-b44f-91e76529f31b service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:37:47.937881 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee589e17-9895-4b13-b44f-91e76529f31b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:37:47.946917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee589e17-9895-4b13-b44f-91e76529f31b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:37:47.948360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee589e17-9895-4b13-b44f-91e76529f31b service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:37:47.957848 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1454/2907] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:37:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:37:54.977913 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:37:57.953415 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:40:01.469726 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef019ce4-70de-4ff5-90fd-cae015199272 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:40:01.487514 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef019ce4-70de-4ff5-90fd-cae015199272 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:40:01.492824 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:40:01.493407 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:40:01.493688 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:40:01.494579 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:40:01.495991 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:40:01.509462 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:40:01.511102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:40:01.522948 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1454/2908] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:40:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:40:11.518090 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:40:38.370728 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee589e17-9895-4b13-b44f-91e76529f31b service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:40:38.372117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ee589e17-9895-4b13-b44f-91e76529f31b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:40:38.440505 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee589e17-9895-4b13-b44f-91e76529f31b service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:40:38.445389 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:40:38.446010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:40:38.446288 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:40:38.446554 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:40:38.448263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:40:38.460010 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:40:38.461843 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:40:38.472384 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1455/2909] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:40:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:40:48.467385 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=32 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:41:25.884901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:41:25.886685 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:41:25.901007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81897658-5648-4cc5-bc17-a5d47b14d8a9 service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:41:25.906409 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a157c-1061-4350-9c98-37226dc90179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:41:25.908037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a157c-1061-4350-9c98-37226dc90179 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:41:25.908322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a157c-1061-4350-9c98-37226dc90179 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:41:25.908596 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a157c-1061-4350-9c98-37226dc90179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:41:25.916094 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a157c-1061-4350-9c98-37226dc90179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:41:25.917656 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a157c-1061-4350-9c98-37226dc90179 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:41:25.926302 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1455/2910] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:41:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:41:35.922385 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:41:53.561874 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:41:53.563319 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:41:53.574914 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32c0cb-6772-4910-8f45-7fd56d570eb4 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:41:53.578995 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de2516-1d56-41ea-9a84-a050b09922cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:41:53.579582 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de2516-1d56-41ea-9a84-a050b09922cd service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:41:53.579859 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de2516-1d56-41ea-9a84-a050b09922cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:41:53.580133 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de2516-1d56-41ea-9a84-a050b09922cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:41:53.588748 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de2516-1d56-41ea-9a84-a050b09922cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:41:53.590243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de2516-1d56-41ea-9a84-a050b09922cd service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:41:53.617961 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1456/2911] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:41:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:42:03.615379 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=13 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:42:13.086532 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a157c-1061-4350-9c98-37226dc90179 service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:42:13.095699 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a157c-1061-4350-9c98-37226dc90179 service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:42:13.099228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cfe31-5e52-4196-9953-fbd04aee8dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:42:13.099810 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cfe31-5e52-4196-9953-fbd04aee8dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:42:13.100084 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cfe31-5e52-4196-9953-fbd04aee8dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:42:13.100356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cfe31-5e52-4196-9953-fbd04aee8dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:42:13.110983 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0cfe31-5e52-4196-9953-fbd04aee8dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:42:13.112396 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0cfe31-5e52-4196-9953-fbd04aee8dd9 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:42:13.122965 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1456/2912] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:42:14.297976 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de2516-1d56-41ea-9a84-a050b09922cd service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:42:14.307303 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de2516-1d56-41ea-9a84-a050b09922cd service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:42:14.310857 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f0f41-4f03-4033-b503-2c350a591aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:42:14.311953 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f0f41-4f03-4033-b503-2c350a591aba service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:42:14.312360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f0f41-4f03-4033-b503-2c350a591aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:42:14.312735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f0f41-4f03-4033-b503-2c350a591aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:42:14.324230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f0f41-4f03-4033-b503-2c350a591aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:42:14.325633 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f0f41-4f03-4033-b503-2c350a591aba service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:42:14.336990 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1457/2913] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:42:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:42:23.118204 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:42:24.332167 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:43:41.731590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cfe31-5e52-4196-9953-fbd04aee8dd9 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:43:41.740539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cfe31-5e52-4196-9953-fbd04aee8dd9 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:43:41.744136 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633e734-55f1-44ce-a8f2-27835ada642f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:43:41.744719 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633e734-55f1-44ce-a8f2-27835ada642f service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:43:41.744993 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633e734-55f1-44ce-a8f2-27835ada642f service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:43:41.745261 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633e734-55f1-44ce-a8f2-27835ada642f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:43:41.753434 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633e734-55f1-44ce-a8f2-27835ada642f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:43:41.754890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633e734-55f1-44ce-a8f2-27835ada642f service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:43:41.763165 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1457/2914] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:43:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:43:51.759463 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:45:42.358715 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f0f41-4f03-4033-b503-2c350a591aba service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:45:42.418759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f0f41-4f03-4033-b503-2c350a591aba service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:45:42.423802 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4be8e-3d14-4933-a36c-bce1308a8944 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:45:42.424399 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4be8e-3d14-4933-a36c-bce1308a8944 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:45:42.424676 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4be8e-3d14-4933-a36c-bce1308a8944 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:45:42.424946 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4be8e-3d14-4933-a36c-bce1308a8944 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:45:42.433893 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4be8e-3d14-4933-a36c-bce1308a8944 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:45:42.435463 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4be8e-3d14-4933-a36c-bce1308a8944 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:45:42.445072 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1458/2915] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:45:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:45:45.584228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633e734-55f1-44ce-a8f2-27835ada642f service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:45:45.637759 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633e734-55f1-44ce-a8f2-27835ada642f service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:45:45.644662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:45:45.645334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:45:45.645615 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:45:45.645890 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:45:45.647816 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:45:45.691022 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:45:45.692751 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:45:45.708243 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1458/2916] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:45:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:45:52.440916 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=27 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:45:55.701087 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=27 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:46:26.958766 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4be8e-3d14-4933-a36c-bce1308a8944 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:46:27.029315 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4be8e-3d14-4933-a36c-bce1308a8944 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:46:27.034646 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:46:27.035234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:46:27.035509 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:46:27.035778 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:46:27.037573 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:46:27.113039 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:46:27.114616 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:46:27.132626 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1459/2917] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:46:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:46:37.130109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=69 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:46:52.989622 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:46:52.991315 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:46:53.067777 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402989cb-9e84-4312-b310-0180a301aaaa service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:46:53.071441 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab8e7-e3ee-4016-bea9-7a7d4db8652e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:46:53.072043 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab8e7-e3ee-4016-bea9-7a7d4db8652e service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:46:53.072322 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab8e7-e3ee-4016-bea9-7a7d4db8652e service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:46:53.072595 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab8e7-e3ee-4016-bea9-7a7d4db8652e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:46:53.074523 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1cbab8e7-e3ee-4016-bea9-7a7d4db8652e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:46:53.144708 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbab8e7-e3ee-4016-bea9-7a7d4db8652e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:46:53.146159 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbab8e7-e3ee-4016-bea9-7a7d4db8652e service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:46:53.155735 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1459/2918] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:46:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:47:03.151616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=62 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:47:14.234681 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:47:14.236290 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:47:14.311230 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6fe64-1187-40ad-8d78-d8adb15f0ade service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:47:14.314735 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:47:14.315334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:47:14.315618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:47:14.315885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:47:14.317643 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:47:14.331224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:47:14.332617 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:47:14.343897 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1460/2919] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:47:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:47:15.519042 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab8e7-e3ee-4016-bea9-7a7d4db8652e service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:47:15.528318 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab8e7-e3ee-4016-bea9-7a7d4db8652e service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:47:15.532102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:47:15.532673 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:47:15.532950 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:47:15.533220 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:47:15.535031 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:47:15.548051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:47:15.549449 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:47:15.560519 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1460/2920] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:47:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:47:24.339041 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=35 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:47:25.555781 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:48:47.087042 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:48:47.088618 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:48:47.098962 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be1a99-e9ce-4628-addd-f433f378ff59 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:48:47.104822 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:48:47.105417 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:48:47.105690 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:48:47.105984 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:48:47.108001 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:48:47.117556 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:48:47.119019 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:48:47.126941 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1461/2921] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:48:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:48:57.123346 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:50:48.347493 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:50:48.348869 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:50:48.427384 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3216976-7704-4f29-abf5-38f7c4d7c456 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:50:48.432975 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d477d-58c7-43d8-af56-1006e0e9dc8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:50:48.433773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d477d-58c7-43d8-af56-1006e0e9dc8f service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:50:48.434227 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d477d-58c7-43d8-af56-1006e0e9dc8f service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:50:48.434498 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d477d-58c7-43d8-af56-1006e0e9dc8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:50:48.436508 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-047d477d-58c7-43d8-af56-1006e0e9dc8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:50:48.448246 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d477d-58c7-43d8-af56-1006e0e9dc8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:50:48.449793 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d477d-58c7-43d8-af56-1006e0e9dc8f service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:50:48.459334 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1461/2922] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:50:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:50:58.455294 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=37 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:51:26.258663 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:51:26.260250 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:51:26.275827 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0457-ea0b-4b64-82a7-bdd15a54d478 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:51:26.281342 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b38254-3a78-4007-81fc-25f7d0874b10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:51:26.281945 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b38254-3a78-4007-81fc-25f7d0874b10 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:51:26.282234 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b38254-3a78-4007-81fc-25f7d0874b10 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:51:26.282502 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b38254-3a78-4007-81fc-25f7d0874b10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:51:26.290710 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b38254-3a78-4007-81fc-25f7d0874b10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:51:26.292198 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b38254-3a78-4007-81fc-25f7d0874b10 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:51:26.301487 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1462/2923] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:51:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:51:36.296834 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:51:45.505739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d477d-58c7-43d8-af56-1006e0e9dc8f service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:51:45.515504 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d477d-58c7-43d8-af56-1006e0e9dc8f service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:51:45.519141 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed2a76-bb63-4e0c-9b32-339fa59a5530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:51:45.519745 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed2a76-bb63-4e0c-9b32-339fa59a5530 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:51:45.520051 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed2a76-bb63-4e0c-9b32-339fa59a5530 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:51:45.520326 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed2a76-bb63-4e0c-9b32-339fa59a5530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:51:45.522276 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-62ed2a76-bb63-4e0c-9b32-339fa59a5530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:51:45.607041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed2a76-bb63-4e0c-9b32-339fa59a5530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:51:45.608481 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed2a76-bb63-4e0c-9b32-339fa59a5530 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:51:45.628046 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1462/2924] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:51:48.760190 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b38254-3a78-4007-81fc-25f7d0874b10 service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:51:48.778084 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b38254-3a78-4007-81fc-25f7d0874b10 service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:51:48.787829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:51:48.787829 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:51:48.788055 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:51:48.788173 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:51:48.791651 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:51:48.809590 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:51:48.811675 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:51:48.831122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1463/2925] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:51:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:51:51.988086 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed2a76-bb63-4e0c-9b32-339fa59a5530 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:51:52.001715 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed2a76-bb63-4e0c-9b32-339fa59a5530 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:51:52.009476 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874304a-3132-434a-9fc6-380807831ef9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:51:52.010134 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874304a-3132-434a-9fc6-380807831ef9 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:51:52.010417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874304a-3132-434a-9fc6-380807831ef9 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:51:52.010850 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874304a-3132-434a-9fc6-380807831ef9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:51:52.012655 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0874304a-3132-434a-9fc6-380807831ef9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:51:52.026546 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874304a-3132-434a-9fc6-380807831ef9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:51:52.028697 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874304a-3132-434a-9fc6-380807831ef9 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:51:52.045542 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1463/2926] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:51:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:51:52.069949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:51:52.072628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:51:52.087006 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94687b8a-a10a-42ba-9952-7105008f15a8 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:51:52.092028 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:51:52.092689 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:51:52.093011 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:51:52.093285 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] PATH_INFO: `/v3/users/0c3f1c1971df49e195a998d55834ff48` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:51:52.097799 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c3f1c1971df49e195a998d55834ff48)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:51:52.099877 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:51:52.149884 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c3f1c1971df49e195a998d55834ff48'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.159475 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c3f1c1971df49e195a998d55834ff48'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.160154 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c3f1c1971df49e195a998d55834ff48'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.160876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c3f1c1971df49e195a998d55834ff48'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.168119 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c3f1c1971df49e195a998d55834ff48'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.186752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1464/2927] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:51:52 2022] DELETE /identity/v3/users/0c3f1c1971df49e195a998d55834ff48 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:51:52.383638 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874304a-3132-434a-9fc6-380807831ef9 service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:51:52.395629 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874304a-3132-434a-9fc6-380807831ef9 service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:51:52.453704 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:51:52.454367 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:51:52.454643 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:51:52.454939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] PATH_INFO: `/v3/projects/9dc6882f51ef41cd9d276e79a85bd868` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:51:52.456653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc6882f51ef41cd9d276e79a85bd868)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:51:52.458703 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:51:52.505539 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc6882f51ef41cd9d276e79a85bd868', 'request_id': 'req-33e24cca-f164-4d65-b39a-a4e36c77bcd8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.509373 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:51:52.511865 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc6882f51ef41cd9d276e79a85bd868', 'request_id': 'req-33e24cca-f164-4d65-b39a-a4e36c77bcd8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.519667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc6882f51ef41cd9d276e79a85bd868', 'request_id': 'req-33e24cca-f164-4d65-b39a-a4e36c77bcd8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.531962 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] The token cache is being invalidate because project 9dc6882f51ef41cd9d276e79a85bd868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:51:52.532247 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.532887 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.534783 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1464/2928] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:51:52 2022] DELETE /identity/v3/projects/9dc6882f51ef41cd9d276e79a85bd868 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:51:52.538885 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:51:52.540322 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:51:52.554109 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc0b28-36d5-4b66-a0c0-03375770b452 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:51:52.583267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:51:52.587939 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:51:52.618917 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:51:52.619732 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:51:52.620029 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:51:52.621085 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] PATH_INFO: `/v3/users/e53ba401eaed45e1a6d614946f5c17b3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:51:52.637092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e53ba401eaed45e1a6d614946f5c17b3)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:51:52.639062 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:51:52.662174 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e24cca-f164-4d65-b39a-a4e36c77bcd8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:51:52.668189 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9168d8bbf8b43299a7748c9faa099b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:51:52.668831 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:51:52.669105 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:51:52.669376 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:51:52.671278 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:51:52.692114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e53ba401eaed45e1a6d614946f5c17b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.694809 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:51:52.700555 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e53ba401eaed45e1a6d614946f5c17b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.701712 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e53ba401eaed45e1a6d614946f5c17b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.703334 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e53ba401eaed45e1a6d614946f5c17b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.711949 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e53ba401eaed45e1a6d614946f5c17b3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:52.729758 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1465/2929] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 05:51:52 2022] DELETE /identity/v3/users/e53ba401eaed45e1a6d614946f5c17b3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:51:52.738565 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:51:52.740267 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:51:52.750002 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1465/2930] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:51:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 05:51:52.936974 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:51:52.938972 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:51:53.008255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10813dcb-b953-43c1-84fe-52f3e6ba6f3d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:51:53.012685 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:51:53.013305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:51:53.013581 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:51:53.013853 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] PATH_INFO: `/v3/projects/2deeb2d3f51d450694d04532abadcdcb` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:51:53.024988 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2deeb2d3f51d450694d04532abadcdcb)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:51:53.027117 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:51:53.072060 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2deeb2d3f51d450694d04532abadcdcb', 'request_id': 'req-4146502a-8e3c-4c03-aab5-9b387a1d750a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:53.078948 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2deeb2d3f51d450694d04532abadcdcb', 'request_id': 'req-4146502a-8e3c-4c03-aab5-9b387a1d750a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:53.085236 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2deeb2d3f51d450694d04532abadcdcb', 'request_id': 'req-4146502a-8e3c-4c03-aab5-9b387a1d750a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:53.097165 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] The token cache is being invalidate because project 2deeb2d3f51d450694d04532abadcdcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 05:51:53.097455 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:53.098122 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 05:51:53.099573 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1466/2931] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 05:51:52 2022] DELETE /identity/v3/projects/2deeb2d3f51d450694d04532abadcdcb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 05:52:02.745418 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=2,keystone:SELECT=61,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:52:03.093610 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=10,keystone:SELECT=67,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:52:14.874862 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:52:14.876881 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:52:14.932538 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8519dc-2223-442d-9163-d0e17c6c8439 service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:52:14.979074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f904e45-2952-436f-8ecf-9cf5f5dcc403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:52:14.979667 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f904e45-2952-436f-8ecf-9cf5f5dcc403 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:52:14.979945 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f904e45-2952-436f-8ecf-9cf5f5dcc403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:52:14.980216 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f904e45-2952-436f-8ecf-9cf5f5dcc403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:52:14.982209 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f904e45-2952-436f-8ecf-9cf5f5dcc403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:52:15.050130 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f904e45-2952-436f-8ecf-9cf5f5dcc403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:52:15.051554 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f904e45-2952-436f-8ecf-9cf5f5dcc403 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:52:15.063442 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1466/2932] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:52:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:52:16.173543 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:52:16.175038 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:52:16.187112 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146502a-8e3c-4c03-aab5-9b387a1d750a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:52:16.190890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:52:16.191484 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:52:16.191759 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:52:16.192025 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:52:16.193694 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:52:16.207224 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:52:16.208641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:52:16.219255 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1467/2933] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:52:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:52:25.057429 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=74 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:52:26.214405 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:53:52.703163 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f904e45-2952-436f-8ecf-9cf5f5dcc403 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:53:52.712549 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f904e45-2952-436f-8ecf-9cf5f5dcc403 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:53:52.716074 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:53:52.716662 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:53:52.716936 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:53:52.717205 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:53:52.718914 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:53:52.728162 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:53:52.729559 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:53:52.738228 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1467/2934] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:53:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:54:02.734058 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:55:54.337331 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:55:54.338775 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:55:54.467624 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828279e-0039-43ce-bd99-d93d656af8fe service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:55:54.472754 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:55:54.473350 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:55:54.473628 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:55:54.473899 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:55:54.475641 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:55:54.487238 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:55:54.488792 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:55:54.498564 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1468/2935] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:55:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:56:04.494263 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=58 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:56:55.961592 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:56:55.963032 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:56:56.066540 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92794bc-9aa9-43af-838d-e79b0153d425 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:56:56.070166 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:56:56.070760 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:56:56.071037 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:56:56.071307 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:56:56.073015 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:56:56.150437 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 05:56:56.151852 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 05:56:56.169883 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1468/2936] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:56:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 05:57:06.167415 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=83 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 05:57:58.307215 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 05:57:58.308594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:57:58.322978 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a94e15-ba04-4446-9512-04b1f7bbe1dc service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 05:57:58.326594 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5f60d-c670-4b92-9efc-03e206bb7a62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63a04998aafd40cc941283700898c92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 05:57:58.327185 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5f60d-c670-4b92-9efc-03e206bb7a62 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 05:57:58.327460 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5f60d-c670-4b92-9efc-03e206bb7a62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 05:57:58.327729 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5f60d-c670-4b92-9efc-03e206bb7a62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 05:57:58.330056 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-70c5f60d-c670-4b92-9efc-03e206bb7a62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-70c5f60d-c670-4b92-9efc-03e206bb7a62 service cinder] Could not find project: 9dc6882f51ef41cd9d276e79a85bd868.: keystone.exception.ProjectNotFound: Could not find project: 9dc6882f51ef41cd9d276e79a85bd868. Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 21 05:57:58.365318 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:57:58.366793 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 05:57:58.368305 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 05:57:58.369876 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 05:57:58.371401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 9dc6882f51ef41cd9d276e79a85bd868. Sep 21 05:57:58.371401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 21 05:57:58.371401 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1469/2937] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 05:57:58 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 60 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 21 05:58:08.333674 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 06:00:55.644487 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 06:00:55.646041 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:00:55.680391 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4fe91-0a07-4d91-ba40-90f429240904 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 06:00:55.684010 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67cb91-8b16-45fb-b036-ed2ce0c57beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c07403facfaf4c71ab5324af6c44f52a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 06:00:55.684612 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67cb91-8b16-45fb-b036-ed2ce0c57beb service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 06:00:55.684889 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67cb91-8b16-45fb-b036-ed2ce0c57beb service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 06:00:55.685158 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67cb91-8b16-45fb-b036-ed2ce0c57beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 06:00:55.686907 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3e67cb91-8b16-45fb-b036-ed2ce0c57beb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:00:55.749122 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67cb91-8b16-45fb-b036-ed2ce0c57beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 06:00:55.750585 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67cb91-8b16-45fb-b036-ed2ce0c57beb service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 06:00:55.760653 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1469/2938] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 06:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 06:00:58.830427 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5f60d-c670-4b92-9efc-03e206bb7a62 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 06:00:58.831929 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-70c5f60d-c670-4b92-9efc-03e206bb7a62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:00:58.869727 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5f60d-c670-4b92-9efc-03e206bb7a62 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 06:00:58.877360 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f94b6-6a05-413d-bba7-69d6f861cf28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 06:00:58.877890 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f94b6-6a05-413d-bba7-69d6f861cf28 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 06:00:58.878354 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f94b6-6a05-413d-bba7-69d6f861cf28 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 06:00:58.878630 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f94b6-6a05-413d-bba7-69d6f861cf28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 06:00:58.882935 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-461f94b6-6a05-413d-bba7-69d6f861cf28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:00:58.914063 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f94b6-6a05-413d-bba7-69d6f861cf28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 06:00:58.915683 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f94b6-6a05-413d-bba7-69d6f861cf28 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 06:00:58.932609 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1470/2939] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 06:00:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 06:01:00.336109 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67cb91-8b16-45fb-b036-ed2ce0c57beb service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 06:01:00.348725 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67cb91-8b16-45fb-b036-ed2ce0c57beb service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 06:01:00.353898 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f9536-b214-4b74-bfc7-efe5144fa2c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 06:01:00.354600 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f9536-b214-4b74-bfc7-efe5144fa2c1 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 06:01:00.355213 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f9536-b214-4b74-bfc7-efe5144fa2c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 06:01:00.355555 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f9536-b214-4b74-bfc7-efe5144fa2c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 06:01:00.365982 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f9536-b214-4b74-bfc7-efe5144fa2c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 06:01:00.368066 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f9536-b214-4b74-bfc7-efe5144fa2c1 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 06:01:00.382591 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1470/2940] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 06:01:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 06:01:02.020337 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f94b6-6a05-413d-bba7-69d6f861cf28 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 06:01:02.031027 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f94b6-6a05-413d-bba7-69d6f861cf28 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 06:01:02.036559 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3ab66-2a2f-4b4f-b067-489bf92be8b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 06:01:02.037158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3ab66-2a2f-4b4f-b067-489bf92be8b3 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 06:01:02.037434 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3ab66-2a2f-4b4f-b067-489bf92be8b3 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 06:01:02.037704 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3ab66-2a2f-4b4f-b067-489bf92be8b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 06:01:02.039466 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-38b3ab66-2a2f-4b4f-b067-489bf92be8b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:01:02.051353 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3ab66-2a2f-4b4f-b067-489bf92be8b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 06:01:02.052901 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3ab66-2a2f-4b4f-b067-489bf92be8b3 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 06:01:02.063176 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1471/2941] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 06:01:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 06:01:02.082941 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f9536-b214-4b74-bfc7-efe5144fa2c1 service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 06:01:02.084648 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-574f9536-b214-4b74-bfc7-efe5144fa2c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:01:02.099590 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f9536-b214-4b74-bfc7-efe5144fa2c1 service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 06:01:02.108329 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 06:01:02.109007 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 06:01:02.109235 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 06:01:02.109567 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] PATH_INFO: `/v3/users/9d05e7bf6ff04193849987316fcedb81` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 06:01:02.111435 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d05e7bf6ff04193849987316fcedb81)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 06:01:02.113270 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 06:01:02.159773 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d05e7bf6ff04193849987316fcedb81'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.160417 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d05e7bf6ff04193849987316fcedb81'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.167739 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d05e7bf6ff04193849987316fcedb81'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.168399 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d05e7bf6ff04193849987316fcedb81'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.175466 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d05e7bf6ff04193849987316fcedb81'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.192383 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1471/2942] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 06:01:02 2022] DELETE /identity/v3/users/9d05e7bf6ff04193849987316fcedb81 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 06:01:02.363725 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3ab66-2a2f-4b4f-b067-489bf92be8b3 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 06:01:02.374282 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3ab66-2a2f-4b4f-b067-489bf92be8b3 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 06:01:02.431243 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 06:01:02.431875 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 06:01:02.432158 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 06:01:02.432435 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] PATH_INFO: `/v3/projects/8d9fd11e99ac438e8308d62d7637b007` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 06:01:02.433553 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9fd11e99ac438e8308d62d7637b007)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 06:01:02.435482 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 06:01:02.483924 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9fd11e99ac438e8308d62d7637b007', 'request_id': 'req-a3a7f3fb-4c25-4276-873d-eda725622655'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.491719 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9fd11e99ac438e8308d62d7637b007', 'request_id': 'req-a3a7f3fb-4c25-4276-873d-eda725622655'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.497477 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9fd11e99ac438e8308d62d7637b007', 'request_id': 'req-a3a7f3fb-4c25-4276-873d-eda725622655'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.509752 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] The token cache is being invalidate because project 8d9fd11e99ac438e8308d62d7637b007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 06:01:02.510070 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.510731 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.512188 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1472/2943] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 06:01:02 2022] DELETE /identity/v3/projects/8d9fd11e99ac438e8308d62d7637b007 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 06:01:02.516271 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 06:01:02.518405 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:01:02.531748 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bfb8f-87df-4664-bb07-b92582a435e3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 06:01:02.562459 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 06:01:02.593616 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 06:01:02.594291 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 06:01:02.594573 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 06:01:02.594878 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] PATH_INFO: `/v3/users/dfb7d4564524432e8b54ae65f350f3f9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 06:01:02.608766 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfb7d4564524432e8b54ae65f350f3f9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 06:01:02.610838 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 06:01:02.658814 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfb7d4564524432e8b54ae65f350f3f9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.659465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfb7d4564524432e8b54ae65f350f3f9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.666618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfb7d4564524432e8b54ae65f350f3f9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.667245 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfb7d4564524432e8b54ae65f350f3f9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.674191 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfb7d4564524432e8b54ae65f350f3f9'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.691465 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1472/2944] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Sep 21 06:01:02 2022] DELETE /identity/v3/users/dfb7d4564524432e8b54ae65f350f3f9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 06:01:02.856199 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 06:01:02.857657 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:01:02.872016 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7f3fb-4c25-4276-873d-eda725622655 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 06:01:02.919644 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 06:01:02.930108 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493af8396fb3419ea552b2a7642f7a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01b2fa2971c94d418f0122cbda3c72f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 06:01:02.930738 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 06:01:02.931020 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 06:01:02.931296 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] PATH_INFO: `/v3/projects/d554d459a8904812addae8aad925b8a2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 06:01:02.942092 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d554d459a8904812addae8aad925b8a2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 06:01:02.944014 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 06:01:02.988622 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd554d459a8904812addae8aad925b8a2', 'request_id': 'req-899bfe2b-6c39-4777-879b-47962d35c117'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:02.995844 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd554d459a8904812addae8aad925b8a2', 'request_id': 'req-899bfe2b-6c39-4777-879b-47962d35c117'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:03.001881 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd554d459a8904812addae8aad925b8a2', 'request_id': 'req-899bfe2b-6c39-4777-879b-47962d35c117'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:03.013170 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] The token cache is being invalidate because project d554d459a8904812addae8aad925b8a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 06:01:03.013458 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:03.014114 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-899bfe2b-6c39-4777-879b-47962d35c117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 06:01:03.015549 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1473/2945] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Sep 21 06:01:02 2022] DELETE /identity/v3/projects/d554d459a8904812addae8aad925b8a2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 06:01:12.687512 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 06:01:13.010247 dv-cinder-849143-5-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 06:06:07.292489 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 06:06:07.293892 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:06:07.394102 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c987ddf-31fc-4446-9160-385fcf698234 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 06:06:07.451755 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86784c46-a9a6-463b-9943-c0efaae3c962 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3d8a1c8b6640ae9e90ece0bbfbf410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c64d6cb0eb348a6a4a0718a7425d3ed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 06:06:07.452356 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86784c46-a9a6-463b-9943-c0efaae3c962 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 06:06:07.452632 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86784c46-a9a6-463b-9943-c0efaae3c962 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 06:06:07.452901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86784c46-a9a6-463b-9943-c0efaae3c962 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 06:06:07.454618 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-86784c46-a9a6-463b-9943-c0efaae3c962 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 06:06:07.465386 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86784c46-a9a6-463b-9943-c0efaae3c962 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 06:06:07.466973 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86784c46-a9a6-463b-9943-c0efaae3c962 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 06:06:07.477082 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1473/2946] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Sep 21 06:06:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 06:06:17.472901 dv-cinder-849143-5-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=71 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}